-- Logs begin at Wed 2022-11-02 02:54:16 UTC, end at Wed 2022-11-02 04:24:41 UTC. -- Nov 02 03:19:52.951814 dv-cinder-860209-6-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Nov 02 03:19:52.959151 dv-cinder-860209-6-iscsi devstack@keystone.service[81489]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Nov 02 03:19:52.963751 dv-cinder-860209-6-iscsi devstack@keystone.service[81489]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Nov 2 03:19:52 2022] *** Nov 02 03:19:52.963751 dv-cinder-860209-6-iscsi devstack@keystone.service[81489]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Nov 02 03:19:52.963751 dv-cinder-860209-6-iscsi devstack@keystone.service[81489]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Nov 02 03:19:52.963751 dv-cinder-860209-6-iscsi devstack@keystone.service[81489]: nodename: dv-cinder-860209-6-iscsi Nov 02 03:19:52.963751 dv-cinder-860209-6-iscsi devstack@keystone.service[81489]: machine: x86_64 Nov 02 03:19:52.963751 dv-cinder-860209-6-iscsi devstack@keystone.service[81489]: clock source: unix Nov 02 03:19:52.963751 dv-cinder-860209-6-iscsi devstack@keystone.service[81489]: pcre jit disabled Nov 02 03:19:52.963751 dv-cinder-860209-6-iscsi devstack@keystone.service[81489]: detected number of CPU cores: 4 Nov 02 03:19:52.963751 dv-cinder-860209-6-iscsi devstack@keystone.service[81489]: current working directory: / Nov 02 03:19:52.963751 dv-cinder-860209-6-iscsi devstack@keystone.service[81489]: detected binary path: /usr/bin/uwsgi-core Nov 02 03:19:52.963751 dv-cinder-860209-6-iscsi devstack@keystone.service[81489]: your processes number limit is 47680 Nov 02 03:19:52.963751 dv-cinder-860209-6-iscsi devstack@keystone.service[81489]: your memory page size is 4096 bytes Nov 02 03:19:52.963751 dv-cinder-860209-6-iscsi devstack@keystone.service[81489]: detected max file descriptor number: 2048 Nov 02 03:19:52.963751 dv-cinder-860209-6-iscsi devstack@keystone.service[81489]: lock engine: pthread robust mutexes Nov 02 03:19:52.963751 dv-cinder-860209-6-iscsi devstack@keystone.service[81489]: thunder lock: enabled Nov 02 03:19:52.963751 dv-cinder-860209-6-iscsi devstack@keystone.service[81489]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Nov 02 03:19:52.963751 dv-cinder-860209-6-iscsi devstack@keystone.service[81489]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Nov 02 03:19:53.003297 dv-cinder-860209-6-iscsi devstack@keystone.service[81489]: Python main interpreter initialized at 0x557bba06b9c0 Nov 02 03:19:53.003297 dv-cinder-860209-6-iscsi devstack@keystone.service[81489]: python threads support enabled Nov 02 03:19:53.003297 dv-cinder-860209-6-iscsi devstack@keystone.service[81489]: your server socket listen backlog is limited to 100 connections Nov 02 03:19:53.003297 dv-cinder-860209-6-iscsi devstack@keystone.service[81489]: your mercy for graceful operations on workers is 90 seconds Nov 02 03:19:53.003671 dv-cinder-860209-6-iscsi devstack@keystone.service[81489]: mapped 403077 bytes (393 KB) for 2 cores Nov 02 03:19:53.003751 dv-cinder-860209-6-iscsi devstack@keystone.service[81489]: *** Operational MODE: preforking *** Nov 02 03:19:53.004058 dv-cinder-860209-6-iscsi devstack@keystone.service[81489]: *** uWSGI is running in multiple interpreter mode *** Nov 02 03:19:53.004058 dv-cinder-860209-6-iscsi devstack@keystone.service[81489]: spawned uWSGI master process (pid: 81489) Nov 02 03:19:53.004262 dv-cinder-860209-6-iscsi systemd[1]: Started Devstack devstack@keystone.service. Nov 02 03:19:53.006432 dv-cinder-860209-6-iscsi devstack@keystone.service[81489]: spawned uWSGI worker 1 (pid: 81503, cores: 1) Nov 02 03:19:53.006432 dv-cinder-860209-6-iscsi devstack@keystone.service[81489]: spawned uWSGI worker 2 (pid: 81504, cores: 1) Nov 02 03:19:53.006432 dv-cinder-860209-6-iscsi devstack@keystone.service[81489]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Nov 02 03:19:57.234929 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Nov 02 03:19:57.234929 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: from cryptography.utils import int_from_bytes Nov 02 03:19:57.235698 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Nov 02 03:19:57.235698 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: from cryptography.utils import int_from_bytes Nov 02 03:19:57.235847 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Nov 02 03:19:57.235847 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: from cryptography.utils import int_from_bytes Nov 02 03:19:57.235847 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Nov 02 03:19:57.235847 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: from cryptography.utils import int_from_bytes Nov 02 03:19:57.284347 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Nov 02 03:19:57.284536 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Nov 02 03:19:57.284726 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] command line args: [] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Nov 02 03:19:57.284991 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Nov 02 03:19:57.285165 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ================================================================================ {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Nov 02 03:19:57.285415 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Nov 02 03:19:57.285500 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] admin_token = **** {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.285610 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Nov 02 03:19:57.285696 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] config_dir = [] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.285801 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] command line args: [] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Nov 02 03:19:57.285877 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.285986 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] config_source = [] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.286072 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Nov 02 03:19:57.286149 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] debug = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.286225 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ================================================================================ {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Nov 02 03:19:57.286344 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.286648 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] admin_token = **** {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.286762 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] config_dir = [] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.286858 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.286959 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.287054 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.287156 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] config_source = [] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.287256 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.287362 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] debug = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.287463 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.287557 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.287684 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] list_limit = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.287786 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.287890 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] log_config_append = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.288012 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.288131 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.288234 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.288340 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] log_dir = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.288340 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] log_file = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.288530 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.288625 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.288723 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] list_limit = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.288723 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] log_config_append = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.288911 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.288991 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.289073 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.289144 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] log_dir = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.289144 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] log_file = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.289316 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.289316 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.290109 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.290194 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.290313 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.290393 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.290464 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.290528 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.290604 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.290685 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.290756 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.290873 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.290951 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.291102 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.291193 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.291330 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.291413 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.291572 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.291649 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.291800 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.291885 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.292058 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.292144 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.292294 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.292541 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.292624 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.26/identity {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.292732 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] publish_errors = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.292813 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.292961 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.293086 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.293209 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.293348 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.293518 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.26/identity {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.293636 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.293759 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] publish_errors = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.293872 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.293990 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.294110 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] standard_threads = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.294241 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.294365 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.294479 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.294597 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.294710 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.294829 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.294954 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.295068 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] use_journal = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.295068 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] use_json = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.295286 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] standard_threads = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.295405 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] use_stderr = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.295522 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.295690 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] use_syslog = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.295818 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.295932 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.296068 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.296181 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.296295 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] use_journal = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.296295 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] use_json = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.296526 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.296640 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] use_stderr = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.296752 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.296864 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] use_syslog = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.296977 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.297097 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Nov 02 03:19:57.297211 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.297324 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.297437 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.297555 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.297668 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.297788 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.297908 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.298027 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.298142 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.298254 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.298374 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.298495 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.298615 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.298728 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.298840 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.298958 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.299071 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.299193 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.299306 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.299419 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.299531 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.299643 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.299754 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.299883 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.300034 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.300152 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.300271 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.300385 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.300497 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.300617 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.300742 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.300856 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.300968 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.301086 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.301278 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.301401 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.301516 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.301634 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.301748 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.301859 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.301980 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.302094 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.302206 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.302319 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.302431 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.302544 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.302664 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.302784 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.302897 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.303016 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.303130 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.303248 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.303369 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.303483 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.303602 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.303721 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.303834 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.304011 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.304142 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.304256 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.304369 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.304481 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.304620 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.304734 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.304855 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.304975 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.305094 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.305207 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.305324 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.305443 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.305568 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.305683 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.305795 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.305913 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.306032 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.306145 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.306265 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.306265 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.306465 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.306577 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.306690 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.306690 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.306975 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.307090 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.307207 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.307326 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.307439 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.307865 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.308024 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.308024 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.308302 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.308418 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.308536 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.308650 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.308772 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.308891 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.309004 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.309122 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.309236 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.309348 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.309453 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.309549 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.309643 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.309736 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.309831 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.309925 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.310027 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.310132 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.310228 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.310368 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.310493 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.310616 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.310736 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.310856 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.310968 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.311081 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.311202 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.311307 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.311426 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.311541 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.311644 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.311755 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.311868 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.312035 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.312153 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.312284 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.312407 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.312522 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.312644 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.312749 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.312860 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.312983 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.313093 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.313093 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.313305 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.313418 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.313531 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.313531 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.313762 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.313877 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.313989 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.314102 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.314215 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.314335 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.314449 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.314562 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.314674 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.314787 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.314899 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.315020 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.315135 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.315248 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.315361 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.315473 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.315596 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.315722 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.315836 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.315981 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.316097 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.316212 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.316335 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.316448 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.316562 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.316676 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.316795 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.316910 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.317023 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.317144 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.317259 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.317372 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.317485 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.317599 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.317712 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.317833 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.317947 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.318061 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.318174 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.318288 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.318400 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.318591 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.318707 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.318821 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.318939 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.319054 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.319175 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.319299 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.319422 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.319544 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.319658 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.319772 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.319885 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.320047 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.320172 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.320288 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.320402 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.320515 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.320635 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.320749 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.320871 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.320986 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.321099 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.321213 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.321326 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.321440 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.321568 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.321683 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] auth.external = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.321797 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.321916 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.322031 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.322170 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.322295 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.322412 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.322534 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.322658 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] auth.password = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.322772 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.322887 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] auth.token = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.323001 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] auth.external = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.323116 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.323237 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.323353 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.323467 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.323583 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.323697 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.323810 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.323932 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] auth.password = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.324065 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.324179 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] auth.token = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.324294 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.324407 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.324522 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.324648 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.324764 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] credential.caching = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.324878 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.324991 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.325105 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.325219 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.325341 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.325447 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.325564 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.325680 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.325794 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.325908 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.326029 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] credential.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.326145 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.326259 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.326372 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.326486 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.326602 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.326724 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.326840 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.326954 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.327068 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.327182 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.327296 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.327418 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.327533 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.327655 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.327770 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.327883 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.328048 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.328175 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.328298 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.328414 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.328528 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] federation.caching = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.328651 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.328768 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.328891 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.329007 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.329121 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.329235 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.329349 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.329464 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.329591 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.329708 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.329830 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] federation.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.329947 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.330061 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.330183 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.330305 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.330421 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.330541 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.330657 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.330771 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.330893 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.331009 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.331124 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.331238 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.331353 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] identity.caching = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.331466 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.331589 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.331713 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.331828 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.331828 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.332057 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.332172 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.332361 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.332479 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.332593 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.332708 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.332822 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.332937 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.333059 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.333175 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.333289 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.333403 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.333517 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.333632 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.333754 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.333870 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.333985 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.334099 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.334213 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.334334 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.334457 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.334572 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.334686 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.334801 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.334921 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.335037 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.335160 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.335275 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.335389 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.335389 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.335616 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.335731 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.335854 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.335854 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.336115 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.336230 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.336827 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.336930 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.337036 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.337153 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.337267 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.337382 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.337502 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.337618 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.337740 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.337856 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.337970 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.338084 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.338199 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.338313 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.338436 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.338551 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.338665 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.338779 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.338892 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.339006 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.339129 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.339247 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.339361 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.339476 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.339589 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.339711 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.339827 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.339942 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.340076 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.340191 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.340306 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.340432 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.340549 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.340663 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.340777 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.340891 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.341005 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.341127 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.341278 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.341401 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.341515 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.341628 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.341740 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.341740 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.341963 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.342076 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.342195 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.342195 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.342408 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.342408 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.342630 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.342630 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.342841 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.342954 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.343071 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.343192 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.343306 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.343421 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.343533 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.343645 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.user = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.343758 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.343882 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.344013 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.344131 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.344245 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.344360 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.344483 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.344611 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.344726 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.344841 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.344955 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.345069 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.345183 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.345305 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.345427 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.345548 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.345663 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.345777 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.345891 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.346014 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.346129 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.346248 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.346370 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.346485 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.346601 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.346723 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.346839 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.346953 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.347067 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.347181 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.347295 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.347418 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.347534 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.347649 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.347763 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.347877 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.348003 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.348131 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.348247 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.348361 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.348475 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.348589 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.348703 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.348827 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.348827 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.349042 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.349157 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.349271 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.349385 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.349507 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.349622 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.349737 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.349851 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.349965 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.350079 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.350201 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.350316 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.350430 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.350548 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.350662 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.350777 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.350900 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.351015 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.351135 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.351250 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.351363 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.351477 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.351600 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.351715 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.351830 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.352032 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.352675 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.352827 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] resource.caching = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.352960 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.353077 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.353192 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.353306 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.353426 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.353543 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.353665 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.353780 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.353896 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.354010 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.354124 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.354238 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.354360 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.354476 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.354590 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] resource.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.354704 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.354818 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.354933 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.355056 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.355171 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] role.caching = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.355285 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.355399 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] role.driver = sql {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.355513 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.355703 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.355822 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.355937 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.356072 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.356187 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.356302 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.356434 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.356551 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.356665 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.356780 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.356894 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.357010 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] role.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.357123 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.357237 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] role.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.357350 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.357464 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.357577 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.357691 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.357813 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.357928 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.358043 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.358157 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.358278 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.358393 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.358516 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.358633 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.358748 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.358862 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.358862 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.359076 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.359076 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.359302 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.359302 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.359517 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.359631 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.359744 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.359866 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.360034 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.360034 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.360254 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.360368 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.360482 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.360608 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.360725 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.360839 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.360953 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.361315 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.361442 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.361559 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.361673 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.361787 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.361901 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.362015 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.362137 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.362254 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.362368 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.362483 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.362597 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.362711 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.362711 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.362935 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.363049 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.363163 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.363276 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.363391 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.363513 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] token.caching = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.363628 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.363742 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.363856 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.363987 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.364103 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.364230 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.364347 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.364461 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.364575 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.364689 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.364803 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.364925 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.365040 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.365154 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.365268 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.365381 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.365497 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] token.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.365623 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.365738 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.365853 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.365966 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.366080 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.366193 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.366316 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.366432 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.366547 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.366661 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.366775 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.366889 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.367011 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.367135 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.367259 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.367374 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.367489 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.367627 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.367744 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.connection = **** {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.367867 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.368046 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.368167 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.368282 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.368396 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.368510 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.368633 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.368749 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.368890 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.369007 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.369121 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.369245 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.369369 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.369485 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.369609 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.369725 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.369840 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.connection = **** {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.369956 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.370070 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.370213 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.370338 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.370454 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.370569 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.370685 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.370800 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.370915 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.371039 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.371155 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.371270 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.371384 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.371499 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.371614 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.371738 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.371854 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.371983 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.372100 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.372215 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.372329 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81504) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Nov 02 03:19:57.372456 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.372456 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.372456 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.372456 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.372456 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.372456 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.372456 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.372456 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Nov 02 03:19:57.372456 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Nov 02 03:19:57.397268 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.397496 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.397718 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.398295 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.398527 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.398751 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.406501 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.407556 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.435623 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.435775 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.435862 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.436031 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.487824 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.488341 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.536045 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.536281 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.537394 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.537632 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.687836 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.688087 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.688309 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.688525 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.688628 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.688739 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.688821 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.688950 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.689033 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.689230 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.689447 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.689661 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.712635 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.712861 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.712946 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.713079 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.713163 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.713291 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.713373 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.713499 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.713590 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.713705 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.713788 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.713910 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.713992 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.714155 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.724847 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.725313 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.737585 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.737808 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.737987 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.738090 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.738169 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.738242 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.738398 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.738481 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.738620 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.738704 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.738827 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81504) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.738906 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.739038 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.739254 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Nov 02 03:19:57.755339 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.755545 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Nov 02 03:19:57.755741 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.755884 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.755993 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Nov 02 03:19:57.756086 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Nov 02 03:19:57.756303 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.756386 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.756503 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Nov 02 03:19:57.756611 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.756777 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.757041 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.757121 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.757307 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.757388 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.757568 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.767158 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.767437 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.767674 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.767767 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.767975 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.768071 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.768251 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.768534 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.775481 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81504) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 03:19:57.776088 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 03:19:57.778393 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81504) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 03:19:57.778675 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.778956 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.779035 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 03:19:57.779237 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.779319 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.779516 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.779598 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.779776 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.779858 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.780072 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.780167 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.780348 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.780431 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.780616 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.780726 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.780893 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.781000 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.781165 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.781267 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.781438 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.781707 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.803743 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.804456 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.805253 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81504) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 03:19:57.805533 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.805953 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 03:19:57.806236 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.810935 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 03:19:57.811221 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.811504 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.811827 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81504) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 03:19:57.812137 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.812421 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.817919 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 03:19:57.818207 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.819007 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81504) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 03:19:57.819294 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.824091 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81504) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 03:19:57.824383 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.824464 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 03:19:57.824665 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.824746 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.824951 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.825033 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.825223 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.825302 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.825504 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.825594 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.825781 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.825862 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.826056 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.826139 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.826340 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.842194 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.842472 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.842646 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.842925 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.846332 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81504) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 03:19:57.846831 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 03:19:57.849562 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.849855 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.850092 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.850391 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.854168 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81504) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 03:19:57.854655 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 03:19:57.857315 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81504) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 03:19:57.857840 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 03:19:57.860568 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.860868 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.861325 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.861414 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.861635 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.861715 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.861924 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.862007 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.862213 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.862294 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.862502 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.862792 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.878513 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81504) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 03:19:57.878798 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.879064 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.879196 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 03:19:57.879322 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.879487 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.879756 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.880047 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.886800 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.887601 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.888355 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.889190 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.889904 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.890178 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.890742 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.891027 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.892845 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81504) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 03:19:57.893130 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.893514 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.893745 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 03:19:57.893827 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.894029 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.894121 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.894312 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.894594 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.894990 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.905214 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81504) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 03:19:57.905523 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.905882 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.906174 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.906254 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 03:19:57.906467 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.906552 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.906752 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.906834 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.907030 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.907119 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.907428 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.907709 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.908015 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.922436 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81504) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 03:19:57.923520 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 03:19:57.925360 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81504) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 03:19:57.926470 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 03:19:57.928293 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.929417 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.929711 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.931498 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.931906 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81504) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 03:19:57.932214 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.932493 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.933015 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 03:19:57.933300 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.933693 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.938800 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.940103 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.940294 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81504) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 03:19:57.941580 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 03:19:57.943280 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.943567 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.943846 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.944144 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.944480 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.944878 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.945161 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.945438 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.951983 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81504) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 03:19:57.952275 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.952555 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.953331 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 03:19:57.953624 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.953920 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.960277 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81504) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 03:19:57.960558 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.960834 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.960927 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Nov 02 03:19:57.961106 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.961189 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.961891 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.961989 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.962158 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.962241 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.962426 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.962514 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.962700 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.962790 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.962962 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.963043 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.963248 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.963359 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.963516 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.963645 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.963784 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.963914 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.964095 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.964201 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.964382 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.964463 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81504) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.964683 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.964977 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Nov 02 03:19:57.998068 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: WSGI app 0 (mountpoint='') ready in 4 seconds on interpreter 0x557bba06b9c0 pid: 81504 (default app) Nov 02 03:19:57.999679 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: WSGI app 0 (mountpoint='') ready in 4 seconds on interpreter 0x557bba06b9c0 pid: 81503 (default app) Nov 02 03:19:58.001238 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06141e80-5539-488f-8058-ab6adefc8c0e None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:19:58.001523 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06141e80-5539-488f-8058-ab6adefc8c0e None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:19:58.001789 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06141e80-5539-488f-8058-ab6adefc8c0e None None] PATH_INFO: `/v3` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:19:58.005910 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1/1] 10.222.0.26 () {50 vars in 791 bytes} [Wed Nov 2 03:19:57 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:04.759277 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cc065d-a296-479f-9d16-7fe84340df67 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:04.759794 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cc065d-a296-479f-9d16-7fe84340df67 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:04.759898 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cc065d-a296-479f-9d16-7fe84340df67 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:04.764144 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1/2] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:20:04 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:20:04.769168 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248927c1-0564-45dd-ae6f-d8ea19ec9fc7 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:04.769594 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248927c1-0564-45dd-ae6f-d8ea19ec9fc7 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:04.769733 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248927c1-0564-45dd-ae6f-d8ea19ec9fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:04.839448 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: INFO dbcounter [None req-248927c1-0564-45dd-ae6f-d8ea19ec9fc7 None None] Registered counter for database keystone Nov 02 03:20:04.858539 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG oslo_db.sqlalchemy.engines [None req-248927c1-0564-45dd-ae6f-d8ea19ec9fc7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81504) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Nov 02 03:20:04.864920 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writer thread running {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Nov 02 03:20:05.040769 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG passlib.handlers.bcrypt [None req-248927c1-0564-45dd-ae6f-d8ea19ec9fc7 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81504) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Nov 02 03:20:05.041159 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG passlib.handlers.bcrypt [None req-248927c1-0564-45dd-ae6f-d8ea19ec9fc7 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81504) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Nov 02 03:20:05.100532 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-248927c1-0564-45dd-ae6f-d8ea19ec9fc7 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:20:05.102092 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-248927c1-0564-45dd-ae6f-d8ea19ec9fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:20:05.234026 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 2/3] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:20:04 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 466 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Nov 02 03:20:05.718052 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cc065d-a296-479f-9d16-7fe84340df67 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:05.790273 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: INFO dbcounter [None req-a6cc065d-a296-479f-9d16-7fe84340df67 None None] Registered counter for database keystone Nov 02 03:20:05.811088 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG oslo_db.sqlalchemy.engines [None req-a6cc065d-a296-479f-9d16-7fe84340df67 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81503) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:326}} Nov 02 03:20:05.817197 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writer thread running {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Nov 02 03:20:05.936305 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cc065d-a296-479f-9d16-7fe84340df67 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:05.940510 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e52747-e07c-4272-bffb-a70228574f39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:05.941137 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e52747-e07c-4272-bffb-a70228574f39 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:05.941413 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e52747-e07c-4272-bffb-a70228574f39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:05.941685 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e52747-e07c-4272-bffb-a70228574f39 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:05.950363 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e52747-e07c-4272-bffb-a70228574f39 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:06.003405 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: WARNING keystone.common.rbac_enforcer.enforcer [None req-73e52747-e07c-4272-bffb-a70228574f39 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Nov 02 03:20:06.003818 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e52747-e07c-4272-bffb-a70228574f39 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:06.013640 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 2/4] 10.222.0.26 () {58 vars in 1324 bytes} [Wed Nov 2 03:20:05 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Nov 02 03:20:06.019544 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248927c1-0564-45dd-ae6f-d8ea19ec9fc7 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:06.036516 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248927c1-0564-45dd-ae6f-d8ea19ec9fc7 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:06.040949 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6435bd68-78cf-4cce-9eff-0dbef7acaf47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:06.041993 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6435bd68-78cf-4cce-9eff-0dbef7acaf47 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:06.042271 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6435bd68-78cf-4cce-9eff-0dbef7acaf47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:06.042546 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6435bd68-78cf-4cce-9eff-0dbef7acaf47 admin admin] PATH_INFO: `/v3/projects/90ca3cc9c54d44feb93e6ed209b99483` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:06.046291 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6435bd68-78cf-4cce-9eff-0dbef7acaf47 admin admin] RBAC: Authorizing `identity:get_project(project_id=90ca3cc9c54d44feb93e6ed209b99483)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:06.097489 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: WARNING keystone.common.rbac_enforcer.enforcer [None req-6435bd68-78cf-4cce-9eff-0dbef7acaf47 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Nov 02 03:20:06.097827 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6435bd68-78cf-4cce-9eff-0dbef7acaf47 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:06.102236 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 3/5] 10.222.0.26 () {56 vars in 1207 bytes} [Wed Nov 2 03:20:06 2022] GET /identity/v3/projects/90ca3cc9c54d44feb93e6ed209b99483 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:08.196970 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a318883a-f74f-4464-9df9-60db7f8cd69e None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:08.197509 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a318883a-f74f-4464-9df9-60db7f8cd69e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:08.197509 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a318883a-f74f-4464-9df9-60db7f8cd69e None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:08.201581 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 3/6] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:20:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 03:20:08.207044 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b42210-09fd-4dc5-bc08-7e765ea754ca None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:08.207429 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b42210-09fd-4dc5-bc08-7e765ea754ca None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:08.207592 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b42210-09fd-4dc5-bc08-7e765ea754ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:08.227798 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-b9b42210-09fd-4dc5-bc08-7e765ea754ca None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:20:08.229325 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-b9b42210-09fd-4dc5-bc08-7e765ea754ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:20:08.237196 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 4/7] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:20:08 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Nov 02 03:20:08.501455 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a318883a-f74f-4464-9df9-60db7f8cd69e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:08.514111 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a318883a-f74f-4464-9df9-60db7f8cd69e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:08.518318 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf64655-3a8b-4093-a4f2-176bd7fce3fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:08.518917 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf64655-3a8b-4093-a4f2-176bd7fce3fb admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:08.519198 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf64655-3a8b-4093-a4f2-176bd7fce3fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:08.519472 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf64655-3a8b-4093-a4f2-176bd7fce3fb admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:08.527095 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf64655-3a8b-4093-a4f2-176bd7fce3fb admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:08.528915 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf64655-3a8b-4093-a4f2-176bd7fce3fb admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:08.537733 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 4/8] 10.222.0.26 () {58 vars in 1324 bytes} [Wed Nov 2 03:20:08 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 03:20:08.543122 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b42210-09fd-4dc5-bc08-7e765ea754ca None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:08.552196 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b42210-09fd-4dc5-bc08-7e765ea754ca None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:08.557097 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ead491-895c-4d14-a38c-e5bf81506b60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:08.557719 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ead491-895c-4d14-a38c-e5bf81506b60 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:08.557996 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ead491-895c-4d14-a38c-e5bf81506b60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:08.558266 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ead491-895c-4d14-a38c-e5bf81506b60 admin admin] PATH_INFO: `/v3/users/cc903821fbc442d9b4a30f92f10f1b17` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:08.559570 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ead491-895c-4d14-a38c-e5bf81506b60 admin admin] RBAC: Authorizing `identity:get_user(user_id=cc903821fbc442d9b4a30f92f10f1b17)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:08.561400 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ead491-895c-4d14-a38c-e5bf81506b60 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:08.562881 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 5/9] 10.222.0.26 () {56 vars in 1198 bytes} [Wed Nov 2 03:20:08 2022] GET /identity/v3/users/cc903821fbc442d9b4a30f92f10f1b17 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:10.716508 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da98bfa0-2b82-49fc-8bc0-1234fdf67c47 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:10.717009 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da98bfa0-2b82-49fc-8bc0-1234fdf67c47 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:10.717121 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da98bfa0-2b82-49fc-8bc0-1234fdf67c47 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:10.718360 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 5/10] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:20:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:20:10.723617 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a2107-bde2-4820-8f43-af5e7468e74a None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:10.724084 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a2107-bde2-4820-8f43-af5e7468e74a None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:10.724325 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a2107-bde2-4820-8f43-af5e7468e74a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:10.743222 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-b02a2107-bde2-4820-8f43-af5e7468e74a None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:20:10.744399 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-b02a2107-bde2-4820-8f43-af5e7468e74a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:20:10.785454 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb42931-1be5-4883-9169-ee79643494a9 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:10.785658 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb42931-1be5-4883-9169-ee79643494a9 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:10.786113 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb42931-1be5-4883-9169-ee79643494a9 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:10.787422 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 6/11] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:20:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 03:20:10.792693 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2814245f-8075-4a7d-ba33-13dab090dd33 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:10.792983 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2814245f-8075-4a7d-ba33-13dab090dd33 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:10.793261 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2814245f-8075-4a7d-ba33-13dab090dd33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:10.826967 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 6/12] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:20:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 03:20:10.835000 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG passlib.handlers.bcrypt [None req-2814245f-8075-4a7d-ba33-13dab090dd33 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81503) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Nov 02 03:20:10.835389 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG passlib.handlers.bcrypt [None req-2814245f-8075-4a7d-ba33-13dab090dd33 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81503) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Nov 02 03:20:10.884758 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-2814245f-8075-4a7d-ba33-13dab090dd33 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:20:10.886309 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2814245f-8075-4a7d-ba33-13dab090dd33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:20:10.980967 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 7/13] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:20:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 03:20:11.092829 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02a2107-bde2-4820-8f43-af5e7468e74a None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:11.116876 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:11.195572 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02a2107-bde2-4820-8f43-af5e7468e74a None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:11.228516 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33745ba6-16ca-4475-9f44-92d7111158be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:11.229149 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33745ba6-16ca-4475-9f44-92d7111158be None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:11.229429 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33745ba6-16ca-4475-9f44-92d7111158be None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:11.229704 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33745ba6-16ca-4475-9f44-92d7111158be None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:11.240707 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33745ba6-16ca-4475-9f44-92d7111158be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:20:11.243942 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2814245f-8075-4a7d-ba33-13dab090dd33 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:11.250103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-33745ba6-16ca-4475-9f44-92d7111158be None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 03:20:11.250103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:20:11.250103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:20:11.250103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:20:11.250103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:20:11.250103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:20:11.250103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:20:11.250103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:20:11.250103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:20:11.250103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:20:11.250103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:20:11.250103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:20:11.250103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:20:11.250103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:20:11.250103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:20:11.250103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:20:11.251003 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:20:11.251003 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:20:11.251003 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:20:11.251003 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:20:11.251003 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:20:11.251003 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:20:11.251003 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:20:11.251003 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:20:11.251003 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:20:11.251003 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:20:11.251003 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:20:11.251003 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:20:11.251003 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:20:11.251003 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:20:11.251003 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:20:11.251003 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:20:11.252106 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:20:11.252106 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:20:11.252106 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:20:11.252106 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:20:11.252106 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:20:11.252106 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:20:11.252106 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:20:11.252106 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:20:11.252106 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:20:11.252106 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:20:11.252106 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 03:20:11.252106 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:20:11.252106 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 7/14] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Nov 2 03:20:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:20:11.256830 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33745ba6-16ca-4475-9f44-92d7111158be None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:11.337210 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:11.345532 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2814245f-8075-4a7d-ba33-13dab090dd33 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:11.348468 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:11.356302 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33745ba6-16ca-4475-9f44-92d7111158be None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:11.384606 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad71e23-10e8-49a7-850c-297155189c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:11.385685 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad71e23-10e8-49a7-850c-297155189c6f None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:11.385963 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad71e23-10e8-49a7-850c-297155189c6f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:11.386232 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad71e23-10e8-49a7-850c-297155189c6f None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:11.387193 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad71e23-10e8-49a7-850c-297155189c6f None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:11.388639 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad71e23-10e8-49a7-850c-297155189c6f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:11.390027 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531724a3-f6e2-4fa6-a821-a06b993c6fd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:11.390636 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531724a3-f6e2-4fa6-a821-a06b993c6fd0 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:11.390919 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531724a3-f6e2-4fa6-a821-a06b993c6fd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:11.391196 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531724a3-f6e2-4fa6-a821-a06b993c6fd0 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:11.391867 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531724a3-f6e2-4fa6-a821-a06b993c6fd0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:20:11.392201 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531724a3-f6e2-4fa6-a821-a06b993c6fd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:11.393583 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531724a3-f6e2-4fa6-a821-a06b993c6fd0 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:11.400122 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.sql.core [None req-8ad71e23-10e8-49a7-850c-297155189c6f None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Nov 02 03:20:11.400122 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Nov 02 03:20:11.400122 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [parameters: {'id': '1fba29a2226140e3b398b7018f2351e6', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Nov 02 03:20:11.400122 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81503) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Nov 02 03:20:11.406147 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 8/15] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Nov 2 03:20:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:20:11.411645 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531724a3-f6e2-4fa6-a821-a06b993c6fd0 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:11.419136 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-8ad71e23-10e8-49a7-850c-297155189c6f None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Nov 02 03:20:11.419136 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:20:11.419136 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 03:20:11.419136 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 03:20:11.419136 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 03:20:11.419136 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 03:20:11.419136 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 03:20:11.419136 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 03:20:11.419136 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 03:20:11.419136 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application conn.query(q) Nov 02 03:20:11.419136 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 03:20:11.419136 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 03:20:11.419136 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 03:20:11.419136 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application result.read() Nov 02 03:20:11.420115 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 03:20:11.420115 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 03:20:11.420115 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 03:20:11.420115 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 03:20:11.420115 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 03:20:11.420115 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 03:20:11.420115 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 03:20:11.420115 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 03:20:11.420115 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Nov 02 03:20:11.420115 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:20:11.420115 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Nov 02 03:20:11.420115 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:20:11.420115 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:20:11.420115 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Nov 02 03:20:11.420115 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return method(*args, **kwargs) Nov 02 03:20:11.420115 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Nov 02 03:20:11.420115 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return project_ref.to_dict() Nov 02 03:20:11.420115 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 03:20:11.421073 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application next(self.gen) Nov 02 03:20:11.421073 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Nov 02 03:20:11.421073 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application yield resource Nov 02 03:20:11.421073 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 03:20:11.421073 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application next(self.gen) Nov 02 03:20:11.421073 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Nov 02 03:20:11.421073 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self.session.rollback() Nov 02 03:20:11.421073 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Nov 02 03:20:11.421073 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self.force_reraise() Nov 02 03:20:11.421073 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Nov 02 03:20:11.421073 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise self.value Nov 02 03:20:11.421073 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Nov 02 03:20:11.421073 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Nov 02 03:20:11.421073 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Nov 02 03:20:11.421073 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application session.commit() Nov 02 03:20:11.421073 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Nov 02 03:20:11.421073 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Nov 02 03:20:11.421934 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Nov 02 03:20:11.421934 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._prepare_impl() Nov 02 03:20:11.421934 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Nov 02 03:20:11.421934 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self.session.flush() Nov 02 03:20:11.421934 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Nov 02 03:20:11.421934 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._flush(objects) Nov 02 03:20:11.421934 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Nov 02 03:20:11.421934 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Nov 02 03:20:11.421934 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Nov 02 03:20:11.421934 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application compat.raise_( Nov 02 03:20:11.421934 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 03:20:11.421934 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception Nov 02 03:20:11.421934 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Nov 02 03:20:11.421934 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application flush_context.execute() Nov 02 03:20:11.421934 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Nov 02 03:20:11.421934 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rec.execute(self) Nov 02 03:20:11.421934 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Nov 02 03:20:11.422799 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Nov 02 03:20:11.422799 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Nov 02 03:20:11.422799 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application _emit_insert_statements( Nov 02 03:20:11.422799 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Nov 02 03:20:11.422799 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application c = connection._execute_20( Nov 02 03:20:11.422799 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Nov 02 03:20:11.422799 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Nov 02 03:20:11.422799 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Nov 02 03:20:11.422799 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Nov 02 03:20:11.422799 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Nov 02 03:20:11.422799 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ret = self._execute_context( Nov 02 03:20:11.422799 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Nov 02 03:20:11.422799 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Nov 02 03:20:11.422799 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Nov 02 03:20:11.422799 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Nov 02 03:20:11.423565 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 03:20:11.423565 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception Nov 02 03:20:11.423565 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 03:20:11.423565 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 03:20:11.423565 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 03:20:11.423565 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 03:20:11.423565 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 03:20:11.423565 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 03:20:11.423565 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 03:20:11.423565 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application conn.query(q) Nov 02 03:20:11.423565 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 03:20:11.423565 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 03:20:11.423565 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 03:20:11.423565 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application result.read() Nov 02 03:20:11.423565 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 03:20:11.423565 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 03:20:11.423565 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 03:20:11.424417 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 03:20:11.424417 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 03:20:11.424417 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 03:20:11.424417 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 03:20:11.424417 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 03:20:11.424417 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Nov 02 03:20:11.424417 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Nov 02 03:20:11.424417 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [parameters: {'id': '1fba29a2226140e3b398b7018f2351e6', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Nov 02 03:20:11.424417 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Nov 02 03:20:11.424417 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:20:11.424417 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 03:20:11.424417 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:20:11.424417 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:20:11.424417 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Nov 02 03:20:11.424417 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Nov 02 03:20:11.425144 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Nov 02 03:20:11.425144 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Nov 02 03:20:11.425144 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Nov 02 03:20:11.425144 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:20:11.425144 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 03:20:11.425144 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:20:11.425144 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:20:11.425144 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:20:11.425144 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:20:11.425144 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:20:11.425144 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:20:11.425144 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:20:11.425144 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:20:11.425144 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:20:11.425144 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:20:11.425144 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:20:11.425144 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:20:11.426072 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Nov 02 03:20:11.426072 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Nov 02 03:20:11.426072 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:20:11.426072 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:20:11.426072 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Nov 02 03:20:11.426072 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Nov 02 03:20:11.426072 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:20:11.426072 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:20:11.426072 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Nov 02 03:20:11.426072 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.Conflict( Nov 02 03:20:11.426072 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Nov 02 03:20:11.426072 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:20:11.426072 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 8/16] 10.222.0.26 () {60 vars in 1137 bytes} [Wed Nov 2 03:20:11 2022] POST /identity/v3/domains => generated 190 bytes in 178 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Nov 02 03:20:11.426072 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad71e23-10e8-49a7-850c-297155189c6f None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:11.514957 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531724a3-f6e2-4fa6-a821-a06b993c6fd0 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:11.527133 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad71e23-10e8-49a7-850c-297155189c6f None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:11.552200 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a0003-f493-4ee9-8b77-43986e0ce801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:11.553132 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a0003-f493-4ee9-8b77-43986e0ce801 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:11.553527 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a0003-f493-4ee9-8b77-43986e0ce801 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:11.553685 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a0003-f493-4ee9-8b77-43986e0ce801 None admin] PATH_INFO: `/v3/users/cc903821fbc442d9b4a30f92f10f1b17` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:11.554828 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2a0003-f493-4ee9-8b77-43986e0ce801 None admin] RBAC: Authorizing `identity:get_user(user_id=cc903821fbc442d9b4a30f92f10f1b17)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:11.556868 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2a0003-f493-4ee9-8b77-43986e0ce801 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:11.558427 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 9/17] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Nov 2 03:20:11 2022] GET /identity/v3/users/cc903821fbc442d9b4a30f92f10f1b17 => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:20:11.561994 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0747eace-afc6-4ab6-8e8c-175fbabc5b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:11.562850 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0747eace-afc6-4ab6-8e8c-175fbabc5b0c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:11.563128 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0747eace-afc6-4ab6-8e8c-175fbabc5b0c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:11.563402 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0747eace-afc6-4ab6-8e8c-175fbabc5b0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:11.563771 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a0003-f493-4ee9-8b77-43986e0ce801 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:11.578026 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:11.581583 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0747eace-afc6-4ab6-8e8c-175fbabc5b0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:11.583082 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0747eace-afc6-4ab6-8e8c-175fbabc5b0c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:11.599378 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-0747eace-afc6-4ab6-8e8c-175fbabc5b0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:20:11.599378 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:20:11.599378 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:20:11.599378 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:20:11.599378 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:20:11.599378 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:20:11.599378 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:20:11.599378 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:20:11.599378 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:20:11.599378 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:20:11.599378 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:20:11.599378 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:20:11.599378 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 03:20:11.599378 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 03:20:11.599378 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 03:20:11.599378 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 03:20:11.600418 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:20:11.600418 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:20:11.600418 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:20:11.600418 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:20:11.600418 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:20:11.600418 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:20:11.600418 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:20:11.600418 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:20:11.600418 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:20:11.600418 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:20:11.600418 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:20:11.600418 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:20:11.600418 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:20:11.600418 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:20:11.600418 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:20:11.600418 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:20:11.600418 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 03:20:11.601245 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 03:20:11.601245 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:20:11.601245 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:20:11.601245 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 9/18] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:20:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 03:20:11.606341 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0747eace-afc6-4ab6-8e8c-175fbabc5b0c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:11.628164 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:11.666256 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a0003-f493-4ee9-8b77-43986e0ce801 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:11.698912 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515e2698-e23a-4ee4-9ee6-136ef7a01389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:11.699540 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515e2698-e23a-4ee4-9ee6-136ef7a01389 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:11.699816 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515e2698-e23a-4ee4-9ee6-136ef7a01389 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:11.700124 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515e2698-e23a-4ee4-9ee6-136ef7a01389 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:11.701240 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515e2698-e23a-4ee4-9ee6-136ef7a01389 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:11.702681 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515e2698-e23a-4ee4-9ee6-136ef7a01389 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:11.704291 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 10/19] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:20:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:11.705891 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0747eace-afc6-4ab6-8e8c-175fbabc5b0c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:11.709817 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515e2698-e23a-4ee4-9ee6-136ef7a01389 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:11.743157 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52105523-18db-40e1-8ab6-bf18932530ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:11.743762 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52105523-18db-40e1-8ab6-bf18932530ad None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:11.744072 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52105523-18db-40e1-8ab6-bf18932530ad None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:11.744347 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52105523-18db-40e1-8ab6-bf18932530ad None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:11.744998 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52105523-18db-40e1-8ab6-bf18932530ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:20:11.745299 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52105523-18db-40e1-8ab6-bf18932530ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:11.746666 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52105523-18db-40e1-8ab6-bf18932530ad None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:11.763378 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 10/20] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Nov 2 03:20:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:20:11.791221 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:11.813294 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515e2698-e23a-4ee4-9ee6-136ef7a01389 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:11.849012 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb187344-1d0c-4006-a636-1f8b08304aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:11.849747 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb187344-1d0c-4006-a636-1f8b08304aa6 None admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:11.849973 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb187344-1d0c-4006-a636-1f8b08304aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:11.850244 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb187344-1d0c-4006-a636-1f8b08304aa6 None admin] PATH_INFO: `/v3/domains/default/users/cc903821fbc442d9b4a30f92f10f1b17/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:11.852790 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb187344-1d0c-4006-a636-1f8b08304aa6 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=cc903821fbc442d9b4a30f92f10f1b17, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:11.854278 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb187344-1d0c-4006-a636-1f8b08304aa6 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:11.969510 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 11/21] 10.222.0.26 () {58 vars in 1361 bytes} [Wed Nov 2 03:20:11 2022] PUT /identity/v3/domains/default/users/cc903821fbc442d9b4a30f92f10f1b17/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 03:20:14.482163 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a37ce53-795d-4bfd-aee3-54a66f450597 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:14.482686 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a37ce53-795d-4bfd-aee3-54a66f450597 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:14.482686 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a37ce53-795d-4bfd-aee3-54a66f450597 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:14.484393 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 11/22] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:20:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:20:14.490976 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f98cf-70fb-49db-8419-540d4a401f15 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:14.491479 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f98cf-70fb-49db-8419-540d4a401f15 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:14.491593 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62f98cf-70fb-49db-8419-540d4a401f15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:14.521293 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-e62f98cf-70fb-49db-8419-540d4a401f15 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:20:14.522859 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e62f98cf-70fb-49db-8419-540d4a401f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:20:14.594306 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf1166-2b2e-43de-b3a0-251bb28d916e None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:14.594608 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf1166-2b2e-43de-b3a0-251bb28d916e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:14.594867 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1cf1166-2b2e-43de-b3a0-251bb28d916e None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:14.596496 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 12/23] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:20:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:20:14.601668 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f91e88-e0d5-4b1f-9c12-69c46a9f832a None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:14.602512 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f91e88-e0d5-4b1f-9c12-69c46a9f832a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:14.602783 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f91e88-e0d5-4b1f-9c12-69c46a9f832a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:14.627045 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-e7f91e88-e0d5-4b1f-9c12-69c46a9f832a None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:20:14.628263 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e7f91e88-e0d5-4b1f-9c12-69c46a9f832a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:20:14.682079 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 12/24] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:20:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 03:20:14.683510 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3874fc0-4082-4d00-8289-c009b068b16e None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:14.683804 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3874fc0-4082-4d00-8289-c009b068b16e None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:14.684115 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3874fc0-4082-4d00-8289-c009b068b16e None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:14.688832 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 13/25] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:20:14 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:20:14.694564 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb4b32-e314-4926-b792-f7a0aefd0087 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:14.694866 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb4b32-e314-4926-b792-f7a0aefd0087 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:14.695135 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb4b32-e314-4926-b792-f7a0aefd0087 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:14.696890 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 14/26] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:20:14 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:20:14.697760 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b703cf9a-8f36-431b-bf5b-62e2bcaa5926 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:14.699200 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b703cf9a-8f36-431b-bf5b-62e2bcaa5926 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:14.699200 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b703cf9a-8f36-431b-bf5b-62e2bcaa5926 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:14.737657 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-b703cf9a-8f36-431b-bf5b-62e2bcaa5926 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:20:14.739222 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-b703cf9a-8f36-431b-bf5b-62e2bcaa5926 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:20:14.794068 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 13/27] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:20:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 03:20:14.798174 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa1e4a-26fa-492f-913e-8cce0696ab54 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:14.798878 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa1e4a-26fa-492f-913e-8cce0696ab54 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:14.800146 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39aa1e4a-26fa-492f-913e-8cce0696ab54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:14.837287 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-39aa1e4a-26fa-492f-913e-8cce0696ab54 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:20:14.840634 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-39aa1e4a-26fa-492f-913e-8cce0696ab54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:20:14.848788 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:14.885832 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:14.897978 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 15/28] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:20:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 03:20:14.957812 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 14/29] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:20:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 03:20:15.112330 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b703cf9a-8f36-431b-bf5b-62e2bcaa5926 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:15.251178 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b703cf9a-8f36-431b-bf5b-62e2bcaa5926 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:15.286983 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596e2ec6-09c9-4d72-bd09-358c8d3f1c32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:15.288006 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e2ec6-09c9-4d72-bd09-358c8d3f1c32 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:15.288493 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e2ec6-09c9-4d72-bd09-358c8d3f1c32 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:15.288969 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e2ec6-09c9-4d72-bd09-358c8d3f1c32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:15.299675 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596e2ec6-09c9-4d72-bd09-358c8d3f1c32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:15.301562 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596e2ec6-09c9-4d72-bd09-358c8d3f1c32 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:15.308628 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aa1e4a-26fa-492f-913e-8cce0696ab54 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:15.338252 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-596e2ec6-09c9-4d72-bd09-358c8d3f1c32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:20:15.338252 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:20:15.338252 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:20:15.338252 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:20:15.338252 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:20:15.338252 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:20:15.338252 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:20:15.338252 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:20:15.338252 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:20:15.338252 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:20:15.338252 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:20:15.338252 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:20:15.338252 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 03:20:15.338252 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 03:20:15.338252 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 03:20:15.338252 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 03:20:15.339180 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:20:15.339180 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:20:15.339180 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:20:15.339180 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:20:15.339180 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:20:15.339180 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:20:15.339180 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:20:15.339180 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:20:15.339180 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:20:15.339180 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:20:15.339180 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:20:15.339180 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:20:15.339180 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:20:15.339180 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:20:15.339180 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:20:15.339180 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:20:15.339180 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 03:20:15.340055 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 03:20:15.340055 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:20:15.340055 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:20:15.348980 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 16/30] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:20:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:20:15.353019 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596e2ec6-09c9-4d72-bd09-358c8d3f1c32 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:15.437181 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:15.501277 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39aa1e4a-26fa-492f-913e-8cce0696ab54 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:15.509004 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596e2ec6-09c9-4d72-bd09-358c8d3f1c32 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:15.554461 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87b685d-ebdf-477f-b90c-e1de05537210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:15.555150 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b685d-ebdf-477f-b90c-e1de05537210 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:15.555436 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b685d-ebdf-477f-b90c-e1de05537210 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:15.555721 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b685d-ebdf-477f-b90c-e1de05537210 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:15.556405 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87b685d-ebdf-477f-b90c-e1de05537210 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:20:15.556712 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87b685d-ebdf-477f-b90c-e1de05537210 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:15.558103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87b685d-ebdf-477f-b90c-e1de05537210 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:15.558286 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c8687a-7e0c-4d37-bcd4-d7ecb49a1f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:15.559055 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c8687a-7e0c-4d37-bcd4-d7ecb49a1f53 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:15.559556 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c8687a-7e0c-4d37-bcd4-d7ecb49a1f53 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:15.560278 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c8687a-7e0c-4d37-bcd4-d7ecb49a1f53 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:15.561121 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c8687a-7e0c-4d37-bcd4-d7ecb49a1f53 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:15.563211 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c8687a-7e0c-4d37-bcd4-d7ecb49a1f53 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:15.579531 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 17/31] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Nov 2 03:20:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:15.580705 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87b685d-ebdf-477f-b90c-e1de05537210 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:15.580996 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:15.697308 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87b685d-ebdf-477f-b90c-e1de05537210 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:15.711575 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:15.718497 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 15/32] 10.222.0.26 () {60 vars in 1131 bytes} [Wed Nov 2 03:20:15 2022] POST /identity/v3/roles => generated 225 bytes in 410 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:20:15.724600 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c8687a-7e0c-4d37-bcd4-d7ecb49a1f53 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:15.738929 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e18f06-3367-44df-8db9-c2232b653629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:15.740044 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e18f06-3367-44df-8db9-c2232b653629 None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:15.740503 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e18f06-3367-44df-8db9-c2232b653629 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:15.740949 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e18f06-3367-44df-8db9-c2232b653629 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:15.741816 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e18f06-3367-44df-8db9-c2232b653629 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:15.743470 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e18f06-3367-44df-8db9-c2232b653629 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:15.766210 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 18/33] 10.222.0.26 () {60 vars in 1131 bytes} [Wed Nov 2 03:20:15 2022] POST /identity/v3/roles => generated 219 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:20:15.767537 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e18f06-3367-44df-8db9-c2232b653629 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:15.997593 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c8687a-7e0c-4d37-bcd4-d7ecb49a1f53 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:16.017776 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e18f06-3367-44df-8db9-c2232b653629 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:16.077710 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2085e325-914b-4a72-9e50-d05506ad2b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:16.078277 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2085e325-914b-4a72-9e50-d05506ad2b95 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:16.078565 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2085e325-914b-4a72-9e50-d05506ad2b95 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:16.078839 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2085e325-914b-4a72-9e50-d05506ad2b95 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:16.085899 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f74d27-1645-429b-bcdd-adb41b1dc880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:16.086093 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f74d27-1645-429b-bcdd-adb41b1dc880 None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:16.086907 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f74d27-1645-429b-bcdd-adb41b1dc880 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:16.087185 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f74d27-1645-429b-bcdd-adb41b1dc880 None admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:16.087900 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f74d27-1645-429b-bcdd-adb41b1dc880 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:16.089398 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f74d27-1645-429b-bcdd-adb41b1dc880 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:16.097691 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2085e325-914b-4a72-9e50-d05506ad2b95 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:20:16.107378 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-2085e325-914b-4a72-9e50-d05506ad2b95 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 03:20:16.107378 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:20:16.107378 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:20:16.107378 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:20:16.107378 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:20:16.107378 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:20:16.107378 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:20:16.107378 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:20:16.107378 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:20:16.107378 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:20:16.107378 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:20:16.107378 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:20:16.107378 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:20:16.107378 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:20:16.107378 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:20:16.107378 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:20:16.108292 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:20:16.108292 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:20:16.108292 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:20:16.108292 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:20:16.108292 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:20:16.108292 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:20:16.108292 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:20:16.108292 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:20:16.108292 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:20:16.108292 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:20:16.108292 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:20:16.108292 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:20:16.108292 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:20:16.108292 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:20:16.108292 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:20:16.108292 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:20:16.109052 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:20:16.109052 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:20:16.109052 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:20:16.109052 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:20:16.109052 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:20:16.109052 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:20:16.109052 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:20:16.109052 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:20:16.109052 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:20:16.109052 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:20:16.109052 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 03:20:16.109052 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:20:16.111005 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 16/34] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Nov 2 03:20:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 392 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:20:16.112821 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c09d431-f5e5-4b97-be45-2e4840e29169 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:16.113257 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c09d431-f5e5-4b97-be45-2e4840e29169 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:16.113699 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c09d431-f5e5-4b97-be45-2e4840e29169 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:16.115118 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 17/35] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:20:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:20:16.118557 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c09d431-f5e5-4b97-be45-2e4840e29169 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:16.130174 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 19/36] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Nov 2 03:20:15 2022] POST /identity/v3/projects => generated 301 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:20:16.131064 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ab1f8-b07c-44e1-965c-12fcfcc73793 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:16.131298 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ab1f8-b07c-44e1-965c-12fcfcc73793 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:16.132368 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ab1f8-b07c-44e1-965c-12fcfcc73793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:16.154824 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-db5ab1f8-b07c-44e1-965c-12fcfcc73793 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:20:16.156496 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-db5ab1f8-b07c-44e1-965c-12fcfcc73793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:20:16.182165 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:16.203703 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:16.247218 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 20/37] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:20:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 03:20:16.255536 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdc6b91-fedf-427e-8618-ae75219c1787 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:16.255536 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdc6b91-fedf-427e-8618-ae75219c1787 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:16.255536 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdc6b91-fedf-427e-8618-ae75219c1787 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:16.255536 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 21/38] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:20:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:20:16.256270 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f9125-bb79-44e1-bf38-0a7a3c3800b7 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:16.256561 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f9125-bb79-44e1-bf38-0a7a3c3800b7 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:16.256839 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f9125-bb79-44e1-bf38-0a7a3c3800b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:16.260579 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c09d431-f5e5-4b97-be45-2e4840e29169 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:16.303119 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf2999a-231e-4d62-b132-dbbbf7d15199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:16.303763 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf2999a-231e-4d62-b132-dbbbf7d15199 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:16.304187 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf2999a-231e-4d62-b132-dbbbf7d15199 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:16.304473 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf2999a-231e-4d62-b132-dbbbf7d15199 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:16.305821 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf2999a-231e-4d62-b132-dbbbf7d15199 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:20:16.306133 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf2999a-231e-4d62-b132-dbbbf7d15199 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:16.307824 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf2999a-231e-4d62-b132-dbbbf7d15199 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:16.323933 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 18/39] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Nov 2 03:20:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:20:16.333148 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf2999a-231e-4d62-b132-dbbbf7d15199 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:16.403605 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-1c6f9125-bb79-44e1-bf38-0a7a3c3800b7 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:20:16.404934 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-1c6f9125-bb79-44e1-bf38-0a7a3c3800b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:20:16.453244 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf2999a-231e-4d62-b132-dbbbf7d15199 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:16.478782 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:16.491972 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 22/40] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:20:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 03:20:16.495532 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92c5f60-9b9f-45f7-ae84-2a3d51a920d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:16.496197 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c5f60-9b9f-45f7-ae84-2a3d51a920d3 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:16.496480 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c5f60-9b9f-45f7-ae84-2a3d51a920d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:16.496753 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92c5f60-9b9f-45f7-ae84-2a3d51a920d3 None admin] PATH_INFO: `/v3/users/cc903821fbc442d9b4a30f92f10f1b17` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:16.498244 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92c5f60-9b9f-45f7-ae84-2a3d51a920d3 None admin] RBAC: Authorizing `identity:get_user(user_id=cc903821fbc442d9b4a30f92f10f1b17)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:16.499680 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92c5f60-9b9f-45f7-ae84-2a3d51a920d3 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:16.501272 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 19/41] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Nov 2 03:20:16 2022] GET /identity/v3/users/cc903821fbc442d9b4a30f92f10f1b17 => generated 247 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:16.506747 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6f9125-bb79-44e1-bf38-0a7a3c3800b7 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:16.531631 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92c5f60-9b9f-45f7-ae84-2a3d51a920d3 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:16.597243 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:16.613647 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6f9125-bb79-44e1-bf38-0a7a3c3800b7 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:16.641783 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92c5f60-9b9f-45f7-ae84-2a3d51a920d3 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:16.647930 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73057e9-03ca-4b96-bb98-e9a49ea6fc4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:16.648595 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73057e9-03ca-4b96-bb98-e9a49ea6fc4b None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:16.649416 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73057e9-03ca-4b96-bb98-e9a49ea6fc4b None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:16.649686 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73057e9-03ca-4b96-bb98-e9a49ea6fc4b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:16.650810 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73057e9-03ca-4b96-bb98-e9a49ea6fc4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:16.652271 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73057e9-03ca-4b96-bb98-e9a49ea6fc4b None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:16.654001 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 23/42] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:20:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:16.659581 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73057e9-03ca-4b96-bb98-e9a49ea6fc4b None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:16.681835 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e40d11-4221-4957-8202-9a07b29fdcb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:16.682431 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e40d11-4221-4957-8202-9a07b29fdcb7 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:16.682710 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e40d11-4221-4957-8202-9a07b29fdcb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:16.682981 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e40d11-4221-4957-8202-9a07b29fdcb7 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:16.683646 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e40d11-4221-4957-8202-9a07b29fdcb7 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:16.685056 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e40d11-4221-4957-8202-9a07b29fdcb7 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:16.700973 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 20/43] 10.222.0.26 () {60 vars in 1131 bytes} [Wed Nov 2 03:20:16 2022] POST /identity/v3/roles => generated 223 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:20:16.761387 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e40d11-4221-4957-8202-9a07b29fdcb7 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:16.785772 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73057e9-03ca-4b96-bb98-e9a49ea6fc4b None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:16.820613 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:16.824052 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce0e82d-a796-467b-88ee-4342ed52dbea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:16.824535 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce0e82d-a796-467b-88ee-4342ed52dbea None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:16.824821 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce0e82d-a796-467b-88ee-4342ed52dbea None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:16.825096 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce0e82d-a796-467b-88ee-4342ed52dbea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:16.826006 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce0e82d-a796-467b-88ee-4342ed52dbea None admin] RBAC: Adding query filter params (role.id=addc0db6b4ac4362b07fe21e14f89180, scope.domain.id=default, user.id=cc903821fbc442d9b4a30f92f10f1b17) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:20:16.826324 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce0e82d-a796-467b-88ee-4342ed52dbea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:16.827829 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce0e82d-a796-467b-88ee-4342ed52dbea None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:16.835660 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 24/44] 10.222.0.26 () {56 vars in 1322 bytes} [Wed Nov 2 03:20:16 2022] GET /identity/v3/role_assignments?user.id=cc903821fbc442d9b4a30f92f10f1b17&scope.domain.id=default&role.id=addc0db6b4ac4362b07fe21e14f89180 => generated 530 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:20:16.896868 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e40d11-4221-4957-8202-9a07b29fdcb7 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:16.936178 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2decd39a-4aeb-432d-94f9-267d7ad7ebbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:16.936838 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2decd39a-4aeb-432d-94f9-267d7ad7ebbe None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:16.937120 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2decd39a-4aeb-432d-94f9-267d7ad7ebbe None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:16.937394 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2decd39a-4aeb-432d-94f9-267d7ad7ebbe None admin] PATH_INFO: `/v3/domains/default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:16.938651 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2decd39a-4aeb-432d-94f9-267d7ad7ebbe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:16.940150 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2decd39a-4aeb-432d-94f9-267d7ad7ebbe None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:16.941969 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 21/45] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:20:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:20:16.947635 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce0e82d-a796-467b-88ee-4342ed52dbea None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:16.969054 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:17.054811 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce0e82d-a796-467b-88ee-4342ed52dbea None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:17.089155 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234759bc-6f50-43a1-b965-3c4e6629348b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:17.089771 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234759bc-6f50-43a1-b965-3c4e6629348b None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:17.090053 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234759bc-6f50-43a1-b965-3c4e6629348b None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:17.090328 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234759bc-6f50-43a1-b965-3c4e6629348b None admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:17.090980 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234759bc-6f50-43a1-b965-3c4e6629348b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:17.092491 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234759bc-6f50-43a1-b965-3c4e6629348b None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:17.117827 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 25/46] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Nov 2 03:20:16 2022] POST /identity/v3/projects => generated 312 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:20:19.258223 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79ce566-5efb-4264-baa0-e4694c9c91b7 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:19.258766 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79ce566-5efb-4264-baa0-e4694c9c91b7 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:19.258766 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79ce566-5efb-4264-baa0-e4694c9c91b7 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:19.260157 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 22/47] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:20:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:20:19.265291 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b84aefb-d6a3-49f5-b076-1d4e21c7dc97 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:19.265677 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b84aefb-d6a3-49f5-b076-1d4e21c7dc97 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:19.265855 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b84aefb-d6a3-49f5-b076-1d4e21c7dc97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:19.284015 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-6b84aefb-d6a3-49f5-b076-1d4e21c7dc97 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:20:19.285183 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-6b84aefb-d6a3-49f5-b076-1d4e21c7dc97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:20:19.344410 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:19.350584 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 26/48] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:20:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 03:20:19.622149 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79ce566-5efb-4264-baa0-e4694c9c91b7 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:19.721622 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79ce566-5efb-4264-baa0-e4694c9c91b7 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:19.738155 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:19.755377 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d32cf69-1c35-48c8-bdeb-dfaf776641dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:19.756031 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32cf69-1c35-48c8-bdeb-dfaf776641dd None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:19.756315 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32cf69-1c35-48c8-bdeb-dfaf776641dd None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:19.756586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d32cf69-1c35-48c8-bdeb-dfaf776641dd None admin] PATH_INFO: `/v3/domains/default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:19.757681 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d32cf69-1c35-48c8-bdeb-dfaf776641dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:19.759697 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d32cf69-1c35-48c8-bdeb-dfaf776641dd None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:19.761253 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 23/49] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:20:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:19.766728 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b84aefb-d6a3-49f5-b076-1d4e21c7dc97 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:19.862589 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b84aefb-d6a3-49f5-b076-1d4e21c7dc97 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:19.895791 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34ac52b-5b64-4834-82a9-846b5c234d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:19.896426 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34ac52b-5b64-4834-82a9-846b5c234d49 None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:19.896710 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34ac52b-5b64-4834-82a9-846b5c234d49 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:19.896986 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34ac52b-5b64-4834-82a9-846b5c234d49 None admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:19.897644 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34ac52b-5b64-4834-82a9-846b5c234d49 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:19.899139 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34ac52b-5b64-4834-82a9-846b5c234d49 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:19.923523 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 27/50] 10.222.0.26 () {60 vars in 1140 bytes} [Wed Nov 2 03:20:19 2022] POST /identity/v3/projects => generated 298 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:20:22.068002 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5efda91-701a-4d82-8b1e-424c73656d53 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:22.068519 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5efda91-701a-4d82-8b1e-424c73656d53 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:22.068625 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5efda91-701a-4d82-8b1e-424c73656d53 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:22.069886 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 24/51] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:20:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:20:22.075055 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f895a5b-6188-4cce-bd95-4839ca0719ee None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:22.075414 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f895a5b-6188-4cce-bd95-4839ca0719ee None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:22.075615 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f895a5b-6188-4cce-bd95-4839ca0719ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:22.094047 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-1f895a5b-6188-4cce-bd95-4839ca0719ee None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:20:22.095232 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-1f895a5b-6188-4cce-bd95-4839ca0719ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:20:22.140159 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:22.161556 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 28/52] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:20:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 03:20:22.425770 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5efda91-701a-4d82-8b1e-424c73656d53 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:22.525129 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5efda91-701a-4d82-8b1e-424c73656d53 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:22.557632 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8096eddc-903e-4d24-b7ed-d9cb308f69a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:22.558258 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8096eddc-903e-4d24-b7ed-d9cb308f69a3 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:22.558539 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8096eddc-903e-4d24-b7ed-d9cb308f69a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:22.558814 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8096eddc-903e-4d24-b7ed-d9cb308f69a3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:22.559910 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8096eddc-903e-4d24-b7ed-d9cb308f69a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:22.561371 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8096eddc-903e-4d24-b7ed-d9cb308f69a3 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:22.562925 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 25/53] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:20:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:22.568611 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f895a5b-6188-4cce-bd95-4839ca0719ee None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:22.663777 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f895a5b-6188-4cce-bd95-4839ca0719ee None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:22.696694 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3217ccdd-e7b8-4077-8e10-bbf5268f58e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:22.697239 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3217ccdd-e7b8-4077-8e10-bbf5268f58e8 None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:22.697516 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3217ccdd-e7b8-4077-8e10-bbf5268f58e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:22.697788 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3217ccdd-e7b8-4077-8e10-bbf5268f58e8 None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:22.698439 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3217ccdd-e7b8-4077-8e10-bbf5268f58e8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:22.699857 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3217ccdd-e7b8-4077-8e10-bbf5268f58e8 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:22.726145 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 29/54] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Nov 2 03:20:22 2022] POST /identity/v3/users => generated 275 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:20:25.965149 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f40b0-75f7-4170-945a-8ba286d6ba29 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:25.966212 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f40b0-75f7-4170-945a-8ba286d6ba29 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:25.966700 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f40b0-75f7-4170-945a-8ba286d6ba29 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:25.968696 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 26/55] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:20:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:20:25.974863 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca39b1-5899-47be-88fa-9dc684e27c45 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:25.975792 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca39b1-5899-47be-88fa-9dc684e27c45 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:25.976362 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cca39b1-5899-47be-88fa-9dc684e27c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:26.005003 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-2cca39b1-5899-47be-88fa-9dc684e27c45 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:20:26.006571 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-2cca39b1-5899-47be-88fa-9dc684e27c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:20:26.040508 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:26.067475 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700fab1a-c281-4ac8-9592-aabbad5c6c83 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:26.067812 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700fab1a-c281-4ac8-9592-aabbad5c6c83 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:26.068064 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700fab1a-c281-4ac8-9592-aabbad5c6c83 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:26.069662 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 27/56] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:20:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:20:26.076039 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa3077d-471c-4127-92e2-5bf12a043a43 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:26.076327 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa3077d-471c-4127-92e2-5bf12a043a43 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:26.076605 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa3077d-471c-4127-92e2-5bf12a043a43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:26.105665 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-baa3077d-471c-4127-92e2-5bf12a043a43 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:20:26.107272 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-baa3077d-471c-4127-92e2-5bf12a043a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:20:26.118182 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 30/57] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:20:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 03:20:26.179824 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:26.203735 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 28/58] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:20:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Nov 02 03:20:26.275703 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95197a9-24e7-42cd-8c38-fdf1ed24f8e8 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:26.276377 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95197a9-24e7-42cd-8c38-fdf1ed24f8e8 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:26.276883 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95197a9-24e7-42cd-8c38-fdf1ed24f8e8 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:26.278495 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 31/59] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:20:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:20:26.283817 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33edfdc-a2f2-41fa-9cd9-8cba49290119 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:26.284308 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33edfdc-a2f2-41fa-9cd9-8cba49290119 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:26.284728 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33edfdc-a2f2-41fa-9cd9-8cba49290119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:26.311975 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a33edfdc-a2f2-41fa-9cd9-8cba49290119 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:20:26.313387 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a33edfdc-a2f2-41fa-9cd9-8cba49290119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:20:26.407791 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 29/60] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:20:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 03:20:26.431089 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95197a9-24e7-42cd-8c38-fdf1ed24f8e8 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:26.495361 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549203d0-6b1e-4d0c-828a-2ab51d2d87e1 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:26.495870 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549203d0-6b1e-4d0c-828a-2ab51d2d87e1 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:26.496305 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549203d0-6b1e-4d0c-828a-2ab51d2d87e1 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:26.497789 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 30/61] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:20:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 03:20:26.503094 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921eae98-578e-4ff9-b399-606ce0ae451c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:26.503530 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921eae98-578e-4ff9-b399-606ce0ae451c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:26.503965 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921eae98-578e-4ff9-b399-606ce0ae451c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:26.527389 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-921eae98-578e-4ff9-b399-606ce0ae451c None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:20:26.529112 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-921eae98-578e-4ff9-b399-606ce0ae451c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:20:26.578076 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95197a9-24e7-42cd-8c38-fdf1ed24f8e8 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:26.624621 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa888134-8686-4199-ad4e-e317c1cb4103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:26.625264 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa888134-8686-4199-ad4e-e317c1cb4103 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:26.625544 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa888134-8686-4199-ad4e-e317c1cb4103 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:26.625819 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa888134-8686-4199-ad4e-e317c1cb4103 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:26.646255 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa888134-8686-4199-ad4e-e317c1cb4103 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:20:26.654517 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-aa888134-8686-4199-ad4e-e317c1cb4103 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 03:20:26.654517 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:20:26.654517 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:20:26.654517 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:20:26.654517 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:20:26.654517 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:20:26.654517 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:20:26.654517 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:20:26.654517 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:20:26.654517 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:20:26.654517 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:20:26.654517 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:20:26.654517 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:20:26.654517 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:20:26.654517 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:20:26.654517 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:20:26.655458 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:20:26.655458 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:20:26.655458 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:20:26.655458 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:20:26.655458 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:20:26.655458 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:20:26.655458 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:20:26.655458 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:20:26.655458 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:20:26.655458 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:20:26.655458 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:20:26.655458 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:20:26.655458 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:20:26.655458 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:20:26.655458 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:20:26.655458 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:20:26.656243 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:20:26.656243 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:20:26.656243 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:20:26.656243 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:20:26.656243 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:20:26.656243 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:20:26.656243 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:20:26.656243 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:20:26.656243 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:20:26.656243 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:20:26.656243 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 03:20:26.656243 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:20:26.656243 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 32/62] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Nov 2 03:20:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:20:26.656885 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa888134-8686-4199-ad4e-e317c1cb4103 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:26.664437 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 31/63] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:20:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 03:20:26.665713 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921eae98-578e-4ff9-b399-606ce0ae451c None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:26.695015 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:26.720506 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:26.779199 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921eae98-578e-4ff9-b399-606ce0ae451c None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:26.817992 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa888134-8686-4199-ad4e-e317c1cb4103 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:26.821264 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775f3eae-29db-4d70-a837-2e382cd975de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:26.821889 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f3eae-29db-4d70-a837-2e382cd975de None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:26.822174 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f3eae-29db-4d70-a837-2e382cd975de None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:26.822450 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775f3eae-29db-4d70-a837-2e382cd975de None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:26.823116 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775f3eae-29db-4d70-a837-2e382cd975de None admin] RBAC: Adding query filter params (name=member) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:20:26.823418 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775f3eae-29db-4d70-a837-2e382cd975de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:26.824839 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775f3eae-29db-4d70-a837-2e382cd975de None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:26.833478 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 32/64] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Nov 2 03:20:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:26.834628 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5038d1-9f93-4d03-ac38-fd87d52c2939 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:26.834923 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5038d1-9f93-4d03-ac38-fd87d52c2939 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:26.835203 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5038d1-9f93-4d03-ac38-fd87d52c2939 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:26.837407 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 33/65] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:20:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:20:26.840120 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5038d1-9f93-4d03-ac38-fd87d52c2939 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:26.858024 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5060b0d0-0f20-4df2-bb50-9c526c61cc04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:26.858682 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5060b0d0-0f20-4df2-bb50-9c526c61cc04 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:26.858965 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5060b0d0-0f20-4df2-bb50-9c526c61cc04 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:26.859240 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5060b0d0-0f20-4df2-bb50-9c526c61cc04 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:26.871838 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5060b0d0-0f20-4df2-bb50-9c526c61cc04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:20:26.876983 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-5060b0d0-0f20-4df2-bb50-9c526c61cc04 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 03:20:26.876983 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:20:26.876983 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:20:26.876983 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:20:26.876983 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:20:26.876983 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:20:26.876983 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:20:26.876983 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:20:26.876983 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:20:26.876983 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:20:26.876983 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:20:26.876983 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:20:26.876983 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:20:26.876983 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:20:26.876983 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:20:26.876983 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:20:26.878040 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:20:26.878040 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:20:26.878040 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:20:26.878040 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:20:26.878040 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:20:26.878040 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:20:26.878040 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:20:26.878040 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:20:26.878040 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:20:26.878040 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:20:26.878040 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:20:26.878040 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:20:26.878040 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:20:26.878040 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:20:26.878040 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:20:26.878040 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:20:26.879525 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:20:26.879525 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:20:26.879525 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:20:26.879525 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:20:26.879525 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:20:26.879525 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:20:26.879525 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:20:26.879525 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:20:26.879525 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:20:26.879525 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:20:26.879525 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 03:20:26.879525 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:20:26.879525 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 33/66] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Nov 2 03:20:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:20:26.879525 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5060b0d0-0f20-4df2-bb50-9c526c61cc04 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:26.973297 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:26.989133 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5060b0d0-0f20-4df2-bb50-9c526c61cc04 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:27.022782 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf490412-3d4f-4276-a14d-35bc331e1031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:27.023413 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf490412-3d4f-4276-a14d-35bc331e1031 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:27.024313 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf490412-3d4f-4276-a14d-35bc331e1031 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:27.024591 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf490412-3d4f-4276-a14d-35bc331e1031 None admin] PATH_INFO: `/v3/users/80c0c58af04e42a6b66aa1ea61117907` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:27.037348 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf490412-3d4f-4276-a14d-35bc331e1031 None admin] RBAC: Authorizing `identity:get_user(user_id=80c0c58af04e42a6b66aa1ea61117907)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:27.038738 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf490412-3d4f-4276-a14d-35bc331e1031 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:27.040465 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 34/67] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Nov 2 03:20:26 2022] GET /identity/v3/users/80c0c58af04e42a6b66aa1ea61117907 => generated 275 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:27.041578 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f849afeb-1477-4995-98ec-d10d227a9e2a None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:27.041861 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f849afeb-1477-4995-98ec-d10d227a9e2a None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:27.042138 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f849afeb-1477-4995-98ec-d10d227a9e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:27.052415 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5038d1-9f93-4d03-ac38-fd87d52c2939 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:27.059017 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-f849afeb-1477-4995-98ec-d10d227a9e2a None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:20:27.059403 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:27.060226 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f849afeb-1477-4995-98ec-d10d227a9e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:20:27.086980 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4baeac-e141-4b69-b77d-7aee1265f0d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:27.087610 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4baeac-e141-4b69-b77d-7aee1265f0d0 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:27.087894 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4baeac-e141-4b69-b77d-7aee1265f0d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:27.088201 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4baeac-e141-4b69-b77d-7aee1265f0d0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:27.098000 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4baeac-e141-4b69-b77d-7aee1265f0d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:20:27.103373 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-4d4baeac-e141-4b69-b77d-7aee1265f0d0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 03:20:27.103373 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:20:27.103373 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:20:27.103373 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:20:27.103373 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:20:27.103373 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:20:27.103373 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:20:27.103373 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:20:27.103373 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:20:27.103373 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:20:27.103373 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:20:27.103373 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:20:27.103373 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:20:27.103373 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:20:27.103373 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:20:27.103373 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:20:27.104243 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:20:27.104243 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:20:27.104243 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:20:27.104243 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:20:27.104243 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:20:27.104243 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:20:27.104243 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:20:27.104243 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:20:27.104243 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:20:27.104243 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:20:27.104243 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:20:27.104243 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:20:27.104243 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:20:27.104243 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:20:27.104243 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:20:27.104243 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:20:27.105007 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:20:27.105007 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:20:27.105007 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:20:27.105007 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:20:27.105007 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:20:27.105007 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:20:27.105007 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:20:27.105007 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:20:27.105007 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:20:27.105007 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:20:27.105007 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 03:20:27.105007 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:20:27.105007 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 34/68] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Nov 2 03:20:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:20:27.105692 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4baeac-e141-4b69-b77d-7aee1265f0d0 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:27.126296 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 35/69] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:20:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 03:20:27.127314 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f849afeb-1477-4995-98ec-d10d227a9e2a None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:27.211100 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:27.216077 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4baeac-e141-4b69-b77d-7aee1265f0d0 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:27.238014 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f849afeb-1477-4995-98ec-d10d227a9e2a None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:27.251716 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73370340-8596-45b9-b989-b06405862203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:27.252750 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73370340-8596-45b9-b989-b06405862203 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:27.253346 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73370340-8596-45b9-b989-b06405862203 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:27.253728 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73370340-8596-45b9-b989-b06405862203 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:27.254389 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73370340-8596-45b9-b989-b06405862203 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:20:27.254700 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73370340-8596-45b9-b989-b06405862203 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:27.256230 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73370340-8596-45b9-b989-b06405862203 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:27.269346 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 35/70] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Nov 2 03:20:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:27.269346 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73370340-8596-45b9-b989-b06405862203 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:27.279061 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f3c78b-51f2-4ce9-a01e-f7c7a39c3ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:27.279853 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3c78b-51f2-4ce9-a01e-f7c7a39c3ac7 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:27.280164 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3c78b-51f2-4ce9-a01e-f7c7a39c3ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:27.280432 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f3c78b-51f2-4ce9-a01e-f7c7a39c3ac7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:27.290846 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f3c78b-51f2-4ce9-a01e-f7c7a39c3ac7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:20:27.295426 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:27.295752 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-e4f3c78b-51f2-4ce9-a01e-f7c7a39c3ac7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 03:20:27.295752 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:20:27.295752 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:20:27.295752 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:20:27.295752 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:20:27.295752 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:20:27.295752 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:20:27.295752 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:20:27.295752 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:20:27.295752 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:20:27.295752 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:20:27.295752 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:20:27.295752 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:20:27.295752 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:20:27.295752 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:20:27.295752 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:20:27.296611 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:20:27.296611 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:20:27.296611 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:20:27.296611 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:20:27.296611 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:20:27.296611 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:20:27.296611 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:20:27.296611 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:20:27.296611 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:20:27.296611 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:20:27.296611 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:20:27.296611 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:20:27.296611 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:20:27.296611 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:20:27.296611 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:20:27.296611 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:20:27.297391 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:20:27.297391 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:20:27.297391 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:20:27.297391 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:20:27.297391 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:20:27.297391 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:20:27.297391 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:20:27.297391 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:20:27.297391 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:20:27.297391 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:20:27.297391 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 03:20:27.297391 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:20:27.297391 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 36/71] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Nov 2 03:20:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:20:27.298019 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f3c78b-51f2-4ce9-a01e-f7c7a39c3ac7 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:27.381270 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73370340-8596-45b9-b989-b06405862203 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:27.398023 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f3c78b-51f2-4ce9-a01e-f7c7a39c3ac7 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:27.416364 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99a1f76-2c95-4cfd-b40b-02f48a6c6843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:27.417004 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a1f76-2c95-4cfd-b40b-02f48a6c6843 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:27.417288 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a1f76-2c95-4cfd-b40b-02f48a6c6843 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:27.417606 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99a1f76-2c95-4cfd-b40b-02f48a6c6843 None admin] PATH_INFO: `/v3/projects/958251514b4b46ef908d92c563878b13` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:27.418728 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99a1f76-2c95-4cfd-b40b-02f48a6c6843 None admin] RBAC: Authorizing `identity:get_project(project_id=958251514b4b46ef908d92c563878b13)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:27.420189 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99a1f76-2c95-4cfd-b40b-02f48a6c6843 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:27.421817 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 36/72] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Nov 2 03:20:27 2022] GET /identity/v3/projects/958251514b4b46ef908d92c563878b13 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:27.422603 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99a1f76-2c95-4cfd-b40b-02f48a6c6843 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:27.433091 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05ff17c-3953-493b-aaa0-f3e387590fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:27.433698 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05ff17c-3953-493b-aaa0-f3e387590fbc None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:27.433980 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05ff17c-3953-493b-aaa0-f3e387590fbc None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:27.434249 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05ff17c-3953-493b-aaa0-f3e387590fbc None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:27.434911 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05ff17c-3953-493b-aaa0-f3e387590fbc None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:20:27.435216 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05ff17c-3953-493b-aaa0-f3e387590fbc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:27.436606 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05ff17c-3953-493b-aaa0-f3e387590fbc None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:27.438005 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:27.445804 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 37/73] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Nov 2 03:20:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:27.446603 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05ff17c-3953-493b-aaa0-f3e387590fbc None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:27.519845 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:27.530021 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99a1f76-2c95-4cfd-b40b-02f48a6c6843 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:27.549421 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05ff17c-3953-493b-aaa0-f3e387590fbc None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:27.563522 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def101f4-d2cb-4343-9aab-939d519a204a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:27.564176 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def101f4-d2cb-4343-9aab-939d519a204a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:27.564455 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def101f4-d2cb-4343-9aab-939d519a204a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:27.564729 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def101f4-d2cb-4343-9aab-939d519a204a None admin] PATH_INFO: `/v3/users/cc903821fbc442d9b4a30f92f10f1b17` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:27.565842 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def101f4-d2cb-4343-9aab-939d519a204a None admin] RBAC: Authorizing `identity:get_user(user_id=cc903821fbc442d9b4a30f92f10f1b17)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:27.567274 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def101f4-d2cb-4343-9aab-939d519a204a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:27.569086 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 37/74] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Nov 2 03:20:27 2022] GET /identity/v3/users/cc903821fbc442d9b4a30f92f10f1b17 => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:27.569635 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def101f4-d2cb-4343-9aab-939d519a204a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:27.586039 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feb7fd4-b7b0-4589-9bce-094072d16493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:27.586702 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feb7fd4-b7b0-4589-9bce-094072d16493 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:27.586980 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feb7fd4-b7b0-4589-9bce-094072d16493 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:27.587251 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feb7fd4-b7b0-4589-9bce-094072d16493 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:27.587911 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feb7fd4-b7b0-4589-9bce-094072d16493 None admin] RBAC: Adding query filter params (name=member) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:20:27.588246 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feb7fd4-b7b0-4589-9bce-094072d16493 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:27.589601 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feb7fd4-b7b0-4589-9bce-094072d16493 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:27.597567 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 38/75] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Nov 2 03:20:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:27.598313 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feb7fd4-b7b0-4589-9bce-094072d16493 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:27.655148 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:27.673445 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def101f4-d2cb-4343-9aab-939d519a204a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:27.698145 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feb7fd4-b7b0-4589-9bce-094072d16493 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:27.706472 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d35250-f469-433d-af8c-e7976dbd3448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:27.707107 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d35250-f469-433d-af8c-e7976dbd3448 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:27.707393 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d35250-f469-433d-af8c-e7976dbd3448 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:27.707671 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d35250-f469-433d-af8c-e7976dbd3448 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:27.708906 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d35250-f469-433d-af8c-e7976dbd3448 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:27.710351 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d35250-f469-433d-af8c-e7976dbd3448 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:27.711996 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 38/76] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:20:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:27.712853 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d35250-f469-433d-af8c-e7976dbd3448 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:27.731520 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3b854e-c775-4f3f-ab11-534767c3d47a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:27.732209 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b854e-c775-4f3f-ab11-534767c3d47a None admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:27.732494 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b854e-c775-4f3f-ab11-534767c3d47a None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:27.732776 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3b854e-c775-4f3f-ab11-534767c3d47a None admin] PATH_INFO: `/v3/projects/958251514b4b46ef908d92c563878b13/users/80c0c58af04e42a6b66aa1ea61117907/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:27.734676 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3b854e-c775-4f3f-ab11-534767c3d47a None admin] RBAC: Authorizing `identity:create_grant(project_id=958251514b4b46ef908d92c563878b13, user_id=80c0c58af04e42a6b66aa1ea61117907, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:27.736143 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3b854e-c775-4f3f-ab11-534767c3d47a None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:27.745274 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:27.750163 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 39/77] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Nov 2 03:20:27 2022] PUT /identity/v3/projects/958251514b4b46ef908d92c563878b13/users/80c0c58af04e42a6b66aa1ea61117907/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:20:27.750957 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3b854e-c775-4f3f-ab11-534767c3d47a None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:27.825192 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d35250-f469-433d-af8c-e7976dbd3448 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:27.865212 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2d9960-bd46-4178-8e76-1e54923fcaf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:27.865869 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d9960-bd46-4178-8e76-1e54923fcaf2 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:27.866148 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d9960-bd46-4178-8e76-1e54923fcaf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:27.866958 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2d9960-bd46-4178-8e76-1e54923fcaf2 None admin] PATH_INFO: `/v3/users/80c0c58af04e42a6b66aa1ea61117907` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:27.868271 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2d9960-bd46-4178-8e76-1e54923fcaf2 None admin] RBAC: Authorizing `identity:get_user(user_id=80c0c58af04e42a6b66aa1ea61117907)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:27.869745 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2d9960-bd46-4178-8e76-1e54923fcaf2 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:27.871397 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 39/78] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Nov 2 03:20:27 2022] GET /identity/v3/users/80c0c58af04e42a6b66aa1ea61117907 => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:27.872234 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2d9960-bd46-4178-8e76-1e54923fcaf2 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:27.885410 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3b854e-c775-4f3f-ab11-534767c3d47a None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:27.906451 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:27.929645 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12a3681-c6f1-415e-917b-fdfb38032c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:27.930855 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a3681-c6f1-415e-917b-fdfb38032c24 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:27.931140 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a3681-c6f1-415e-917b-fdfb38032c24 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:27.931413 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12a3681-c6f1-415e-917b-fdfb38032c24 None admin] PATH_INFO: `/v3/projects/958251514b4b46ef908d92c563878b13` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:27.932616 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12a3681-c6f1-415e-917b-fdfb38032c24 None admin] RBAC: Authorizing `identity:get_project(project_id=958251514b4b46ef908d92c563878b13)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:27.934064 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12a3681-c6f1-415e-917b-fdfb38032c24 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:27.935871 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 40/79] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Nov 2 03:20:27 2022] GET /identity/v3/projects/958251514b4b46ef908d92c563878b13 => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:27.940758 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12a3681-c6f1-415e-917b-fdfb38032c24 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:27.983312 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:27.987542 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2d9960-bd46-4178-8e76-1e54923fcaf2 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:28.027578 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74fdff4-ffe9-430a-9721-ed42ec35211a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:28.028412 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74fdff4-ffe9-430a-9721-ed42ec35211a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:28.028698 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74fdff4-ffe9-430a-9721-ed42ec35211a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:28.028978 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74fdff4-ffe9-430a-9721-ed42ec35211a None admin] PATH_INFO: `/v3/users/80c0c58af04e42a6b66aa1ea61117907` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:28.030130 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74fdff4-ffe9-430a-9721-ed42ec35211a None admin] RBAC: Authorizing `identity:get_user(user_id=80c0c58af04e42a6b66aa1ea61117907)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:28.031568 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74fdff4-ffe9-430a-9721-ed42ec35211a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:28.033199 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 40/80] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Nov 2 03:20:27 2022] GET /identity/v3/users/80c0c58af04e42a6b66aa1ea61117907 => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:28.038150 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74fdff4-ffe9-430a-9721-ed42ec35211a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:28.051583 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12a3681-c6f1-415e-917b-fdfb38032c24 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:28.087301 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab2aac0-d93e-406a-8336-f964dbb75dda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:28.087842 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab2aac0-d93e-406a-8336-f964dbb75dda None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:28.088156 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab2aac0-d93e-406a-8336-f964dbb75dda None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:28.088434 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab2aac0-d93e-406a-8336-f964dbb75dda None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:28.089090 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab2aac0-d93e-406a-8336-f964dbb75dda None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:28.090540 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab2aac0-d93e-406a-8336-f964dbb75dda None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:28.124068 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 41/81] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Nov 2 03:20:27 2022] POST /identity/v3/users => generated 289 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:20:28.124068 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab2aac0-d93e-406a-8336-f964dbb75dda None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:28.160449 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74fdff4-ffe9-430a-9721-ed42ec35211a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:28.179397 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:28.219441 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c4e278-567b-4d91-9de7-4f60e7fcec73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:28.220455 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c4e278-567b-4d91-9de7-4f60e7fcec73 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:28.220918 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c4e278-567b-4d91-9de7-4f60e7fcec73 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:28.222112 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c4e278-567b-4d91-9de7-4f60e7fcec73 None admin] PATH_INFO: `/v3/projects/958251514b4b46ef908d92c563878b13` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:28.223758 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c4e278-567b-4d91-9de7-4f60e7fcec73 None admin] RBAC: Authorizing `identity:get_project(project_id=958251514b4b46ef908d92c563878b13)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:28.226621 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c4e278-567b-4d91-9de7-4f60e7fcec73 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:28.229302 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 41/82] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Nov 2 03:20:28 2022] GET /identity/v3/projects/958251514b4b46ef908d92c563878b13 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:28.230860 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c4e278-567b-4d91-9de7-4f60e7fcec73 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:28.251918 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab2aac0-d93e-406a-8336-f964dbb75dda None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:28.283450 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:28.296539 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d3a405-c7d4-4ee0-ac7e-4fc859bca9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:28.297671 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d3a405-c7d4-4ee0-ac7e-4fc859bca9bd None admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:28.298146 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d3a405-c7d4-4ee0-ac7e-4fc859bca9bd None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:28.298709 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d3a405-c7d4-4ee0-ac7e-4fc859bca9bd None admin] PATH_INFO: `/v3/projects/958251514b4b46ef908d92c563878b13/users/cc903821fbc442d9b4a30f92f10f1b17/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:28.301143 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d3a405-c7d4-4ee0-ac7e-4fc859bca9bd None admin] RBAC: Authorizing `identity:create_grant(project_id=958251514b4b46ef908d92c563878b13, user_id=cc903821fbc442d9b4a30f92f10f1b17, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:28.303201 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d3a405-c7d4-4ee0-ac7e-4fc859bca9bd None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:28.322431 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 42/83] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Nov 2 03:20:28 2022] PUT /identity/v3/projects/958251514b4b46ef908d92c563878b13/users/cc903821fbc442d9b4a30f92f10f1b17/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:20:28.323097 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d3a405-c7d4-4ee0-ac7e-4fc859bca9bd None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:28.384181 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c4e278-567b-4d91-9de7-4f60e7fcec73 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:28.431282 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5865d370-b58a-474c-b390-6b0278bd40b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:28.431994 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5865d370-b58a-474c-b390-6b0278bd40b3 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:28.432269 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5865d370-b58a-474c-b390-6b0278bd40b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:28.432550 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5865d370-b58a-474c-b390-6b0278bd40b3 None admin] PATH_INFO: `/v3/projects/e0ec9f4137124973b29cd93847af3932` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:28.433894 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5865d370-b58a-474c-b390-6b0278bd40b3 None admin] RBAC: Authorizing `identity:get_project(project_id=e0ec9f4137124973b29cd93847af3932)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:28.435389 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5865d370-b58a-474c-b390-6b0278bd40b3 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:28.437544 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 42/84] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Nov 2 03:20:28 2022] GET /identity/v3/projects/e0ec9f4137124973b29cd93847af3932 => generated 312 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:28.444114 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5865d370-b58a-474c-b390-6b0278bd40b3 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:28.457537 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d3a405-c7d4-4ee0-ac7e-4fc859bca9bd None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:28.460014 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:28.533857 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9478af-2439-4fe1-b6b5-e7bcc2e951e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:28.533857 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9478af-2439-4fe1-b6b5-e7bcc2e951e5 None admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:28.533857 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9478af-2439-4fe1-b6b5-e7bcc2e951e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:28.533857 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9478af-2439-4fe1-b6b5-e7bcc2e951e5 None admin] PATH_INFO: `/v3/projects/958251514b4b46ef908d92c563878b13/users/80c0c58af04e42a6b66aa1ea61117907/roles/f25d67f31fee4a92b3060e45a3fbe379` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:28.540858 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9478af-2439-4fe1-b6b5-e7bcc2e951e5 None admin] RBAC: Authorizing `identity:create_grant(project_id=958251514b4b46ef908d92c563878b13, user_id=80c0c58af04e42a6b66aa1ea61117907, role_id=f25d67f31fee4a92b3060e45a3fbe379)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:28.542886 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9478af-2439-4fe1-b6b5-e7bcc2e951e5 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:28.562713 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 43/85] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Nov 2 03:20:28 2022] PUT /identity/v3/projects/958251514b4b46ef908d92c563878b13/users/80c0c58af04e42a6b66aa1ea61117907/roles/f25d67f31fee4a92b3060e45a3fbe379 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:20:28.609227 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5865d370-b58a-474c-b390-6b0278bd40b3 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:28.612319 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:28.652585 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf63dee-8bc8-47a2-8927-b88f261da2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:28.653427 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf63dee-8bc8-47a2-8927-b88f261da2f9 None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:28.653820 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf63dee-8bc8-47a2-8927-b88f261da2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:28.654205 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf63dee-8bc8-47a2-8927-b88f261da2f9 None admin] PATH_INFO: `/v3/projects/e0ec9f4137124973b29cd93847af3932/users/80c0c58af04e42a6b66aa1ea61117907/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:28.656803 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf63dee-8bc8-47a2-8927-b88f261da2f9 None admin] RBAC: Authorizing `identity:create_grant(project_id=e0ec9f4137124973b29cd93847af3932, user_id=80c0c58af04e42a6b66aa1ea61117907, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:28.658612 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf63dee-8bc8-47a2-8927-b88f261da2f9 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:28.680719 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 43/86] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Nov 2 03:20:28 2022] PUT /identity/v3/projects/e0ec9f4137124973b29cd93847af3932/users/80c0c58af04e42a6b66aa1ea61117907/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:20:30.344998 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ce1e9-571a-482f-904f-293a94802a70 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:30.345508 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ce1e9-571a-482f-904f-293a94802a70 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:30.345621 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690ce1e9-571a-482f-904f-293a94802a70 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:30.347180 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 44/87] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:20:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:20:30.352324 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac54a4-eb34-44c5-9291-daafd272c628 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:30.352710 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac54a4-eb34-44c5-9291-daafd272c628 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:30.352875 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ac54a4-eb34-44c5-9291-daafd272c628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:30.381305 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a9ac54a4-eb34-44c5-9291-daafd272c628 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:20:30.382514 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a9ac54a4-eb34-44c5-9291-daafd272c628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:20:30.471839 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 44/88] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:20:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 03:20:30.817037 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690ce1e9-571a-482f-904f-293a94802a70 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:30.951664 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690ce1e9-571a-482f-904f-293a94802a70 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:30.997367 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5d354d-a0c4-4840-9d96-50dcd9b70284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:30.998015 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5d354d-a0c4-4840-9d96-50dcd9b70284 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:30.998302 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5d354d-a0c4-4840-9d96-50dcd9b70284 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:30.998582 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5d354d-a0c4-4840-9d96-50dcd9b70284 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:31.016860 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5d354d-a0c4-4840-9d96-50dcd9b70284 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:20:31.022389 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-aa5d354d-a0c4-4840-9d96-50dcd9b70284 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 03:20:31.022389 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:20:31.022389 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:20:31.022389 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:20:31.022389 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:20:31.022389 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:20:31.022389 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:20:31.022389 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:20:31.022389 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:20:31.022389 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:20:31.022389 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:20:31.022389 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:20:31.022389 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:20:31.022389 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:20:31.022389 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:20:31.022389 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:20:31.023279 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:20:31.023279 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:20:31.023279 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:20:31.023279 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:20:31.023279 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:20:31.023279 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:20:31.023279 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:20:31.023279 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:20:31.023279 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:20:31.023279 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:20:31.023279 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:20:31.023279 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:20:31.023279 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:20:31.023279 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:20:31.023279 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:20:31.023279 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:20:31.024637 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:20:31.024637 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:20:31.024637 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:20:31.024637 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:20:31.024637 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:20:31.024637 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:20:31.024637 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:20:31.024637 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:20:31.024637 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:20:31.024637 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:20:31.024637 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 03:20:31.024637 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:20:31.024637 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 45/89] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Nov 2 03:20:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:20:31.030089 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ac54a4-eb34-44c5-9291-daafd272c628 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:31.165823 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:31.176503 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ac54a4-eb34-44c5-9291-daafd272c628 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:31.226553 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982523c2-e153-454c-9469-848b66e88a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:31.227245 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982523c2-e153-454c-9469-848b66e88a22 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:31.227532 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982523c2-e153-454c-9469-848b66e88a22 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:31.227809 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982523c2-e153-454c-9469-848b66e88a22 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:31.228510 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982523c2-e153-454c-9469-848b66e88a22 None admin] RBAC: Adding query filter params (name=member) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:20:31.228816 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982523c2-e153-454c-9469-848b66e88a22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:31.230299 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982523c2-e153-454c-9469-848b66e88a22 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:31.244244 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 45/90] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Nov 2 03:20:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:31.250034 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5d354d-a0c4-4840-9d96-50dcd9b70284 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:31.288068 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:31.342190 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c012cc-552f-4db1-bdba-6463fe7a5eec None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:31.342494 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c012cc-552f-4db1-bdba-6463fe7a5eec None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:31.342760 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c012cc-552f-4db1-bdba-6463fe7a5eec None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:31.344771 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 46/91] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:20:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 03:20:31.350138 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38d71b-4397-430e-876f-a076809db9ff None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:31.350426 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38d71b-4397-430e-876f-a076809db9ff None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:31.350878 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38d71b-4397-430e-876f-a076809db9ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:31.381649 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-2f38d71b-4397-430e-876f-a076809db9ff None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:20:31.383764 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2f38d71b-4397-430e-876f-a076809db9ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:20:31.432071 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5d354d-a0c4-4840-9d96-50dcd9b70284 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:31.485788 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc9ebeb-7def-45af-8316-529fdc76a11c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:31.486437 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc9ebeb-7def-45af-8316-529fdc76a11c None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:31.486959 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc9ebeb-7def-45af-8316-529fdc76a11c None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:31.487240 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc9ebeb-7def-45af-8316-529fdc76a11c None admin] PATH_INFO: `/v3/users/80c0c58af04e42a6b66aa1ea61117907` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:31.488584 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc9ebeb-7def-45af-8316-529fdc76a11c None admin] RBAC: Authorizing `identity:get_user(user_id=80c0c58af04e42a6b66aa1ea61117907)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:31.490020 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc9ebeb-7def-45af-8316-529fdc76a11c None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:31.491761 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 46/92] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Nov 2 03:20:31 2022] GET /identity/v3/users/80c0c58af04e42a6b66aa1ea61117907 => generated 275 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:31.492802 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacb22fb-0f59-4bef-b493-8847006887e1 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:31.493091 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacb22fb-0f59-4bef-b493-8847006887e1 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:31.493353 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacb22fb-0f59-4bef-b493-8847006887e1 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:31.494550 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 47/93] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:20:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:20:31.506932 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacb22fb-0f59-4bef-b493-8847006887e1 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:31.513913 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 47/94] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:20:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 03:20:31.515255 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd47f3-e91a-4b8d-bd7f-e80b053e432d None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:31.515549 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd47f3-e91a-4b8d-bd7f-e80b053e432d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:31.515829 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffd47f3-e91a-4b8d-bd7f-e80b053e432d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:31.564975 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-9ffd47f3-e91a-4b8d-bd7f-e80b053e432d None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:20:31.566187 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9ffd47f3-e91a-4b8d-bd7f-e80b053e432d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:20:31.710627 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:31.713379 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 48/95] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:20:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 03:20:31.734193 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacb22fb-0f59-4bef-b493-8847006887e1 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:31.797205 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b25581-0e90-4f2a-99c2-05405b5cad07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:31.798185 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b25581-0e90-4f2a-99c2-05405b5cad07 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:31.798661 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b25581-0e90-4f2a-99c2-05405b5cad07 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:31.799155 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b25581-0e90-4f2a-99c2-05405b5cad07 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:31.820808 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702d026d-3f49-43e7-a123-0b319146119c None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:31.821095 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702d026d-3f49-43e7-a123-0b319146119c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:31.821611 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702d026d-3f49-43e7-a123-0b319146119c None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:31.823415 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 49/96] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:20:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 03:20:31.828225 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8336903f-2470-4be6-b910-2bd3b43dad78 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:31.828750 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8336903f-2470-4be6-b910-2bd3b43dad78 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:31.829341 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8336903f-2470-4be6-b910-2bd3b43dad78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:31.861044 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-8336903f-2470-4be6-b910-2bd3b43dad78 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:20:31.862826 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8336903f-2470-4be6-b910-2bd3b43dad78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:20:31.880073 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:31.933071 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b25581-0e90-4f2a-99c2-05405b5cad07 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:31.934886 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b25581-0e90-4f2a-99c2-05405b5cad07 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:31.977761 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 50/97] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:20:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 03:20:31.978617 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8336903f-2470-4be6-b910-2bd3b43dad78 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:32.049138 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:32.060454 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 48/98] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Nov 2 03:20:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 03:20:32.066098 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b25581-0e90-4f2a-99c2-05405b5cad07 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:32.175074 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b25581-0e90-4f2a-99c2-05405b5cad07 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:32.182482 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8336903f-2470-4be6-b910-2bd3b43dad78 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:32.221565 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc2fcd6-2f15-483e-8243-3544314ee0c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:32.222233 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1855d7eb-f068-47b2-bde4-e218173852c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:32.222376 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc2fcd6-2f15-483e-8243-3544314ee0c3 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:32.222569 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc2fcd6-2f15-483e-8243-3544314ee0c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:32.222922 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855d7eb-f068-47b2-bde4-e218173852c3 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:32.223164 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc2fcd6-2f15-483e-8243-3544314ee0c3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:32.223285 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855d7eb-f068-47b2-bde4-e218173852c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:32.223460 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1855d7eb-f068-47b2-bde4-e218173852c3 None admin] PATH_INFO: `/v3/projects/958251514b4b46ef908d92c563878b13` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:32.224772 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1855d7eb-f068-47b2-bde4-e218173852c3 None admin] RBAC: Authorizing `identity:get_project(project_id=958251514b4b46ef908d92c563878b13)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:32.227411 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1855d7eb-f068-47b2-bde4-e218173852c3 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:32.229209 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 49/99] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Nov 2 03:20:32 2022] GET /identity/v3/projects/958251514b4b46ef908d92c563878b13 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:32.230557 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdc4dd8-0dd0-40fc-b7f0-28ba3ba05495 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:32.230958 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdc4dd8-0dd0-40fc-b7f0-28ba3ba05495 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:32.231294 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdc4dd8-0dd0-40fc-b7f0-28ba3ba05495 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:32.232798 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 50/100] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:20:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:20:32.234087 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdc4dd8-0dd0-40fc-b7f0-28ba3ba05495 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:32.237272 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc2fcd6-2f15-483e-8243-3544314ee0c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:20:32.242614 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-7fc2fcd6-2f15-483e-8243-3544314ee0c3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 03:20:32.242614 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:20:32.242614 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:20:32.242614 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:20:32.242614 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:20:32.242614 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:20:32.242614 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:20:32.242614 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:20:32.242614 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:20:32.242614 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:20:32.242614 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:20:32.242614 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:20:32.242614 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:20:32.242614 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:20:32.242614 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:20:32.242614 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:20:32.243482 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:20:32.243482 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:20:32.243482 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:20:32.243482 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:20:32.243482 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:20:32.243482 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:20:32.243482 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:20:32.243482 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:20:32.243482 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:20:32.243482 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:20:32.243482 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:20:32.243482 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:20:32.243482 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:20:32.243482 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:20:32.243482 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:20:32.243482 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:20:32.244283 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:20:32.244283 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:20:32.244283 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:20:32.244283 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:20:32.244283 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:20:32.244283 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:20:32.244283 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:20:32.244283 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:20:32.244283 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:20:32.244283 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:20:32.244283 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 03:20:32.244283 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:20:32.246526 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 51/101] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Nov 2 03:20:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:20:32.248606 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc2fcd6-2f15-483e-8243-3544314ee0c3 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:32.262638 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:32.309061 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:32.374037 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdc4dd8-0dd0-40fc-b7f0-28ba3ba05495 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:32.379775 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc2fcd6-2f15-483e-8243-3544314ee0c3 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:32.411680 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf3e55-d62a-438a-bf97-0d66ad88ed73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:32.412352 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf3e55-d62a-438a-bf97-0d66ad88ed73 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:32.412635 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf3e55-d62a-438a-bf97-0d66ad88ed73 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:32.412913 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbf3e55-d62a-438a-bf97-0d66ad88ed73 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:32.416430 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30daeb5f-708f-4d7b-bb52-f9a23cb9b643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:32.417109 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30daeb5f-708f-4d7b-bb52-f9a23cb9b643 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:32.417390 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30daeb5f-708f-4d7b-bb52-f9a23cb9b643 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:32.417662 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30daeb5f-708f-4d7b-bb52-f9a23cb9b643 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:32.418460 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30daeb5f-708f-4d7b-bb52-f9a23cb9b643 None admin] RBAC: Adding query filter params (role.id=7a6df2ce66694822bce7983f7d988da5, scope.project.id=958251514b4b46ef908d92c563878b13, user.id=80c0c58af04e42a6b66aa1ea61117907) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:20:32.418768 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30daeb5f-708f-4d7b-bb52-f9a23cb9b643 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:32.420240 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30daeb5f-708f-4d7b-bb52-f9a23cb9b643 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:32.424100 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbf3e55-d62a-438a-bf97-0d66ad88ed73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:20:32.428452 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 52/102] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Nov 2 03:20:32 2022] GET /identity/v3/role_assignments?user.id=80c0c58af04e42a6b66aa1ea61117907&scope.project.id=958251514b4b46ef908d92c563878b13&role.id=7a6df2ce66694822bce7983f7d988da5 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:32.429087 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-4fbf3e55-d62a-438a-bf97-0d66ad88ed73 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 03:20:32.429087 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:20:32.429087 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:20:32.429087 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:20:32.429087 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:20:32.429087 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:20:32.429087 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:20:32.429087 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:20:32.429087 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:20:32.429087 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:20:32.429087 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:20:32.429087 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:20:32.429087 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:20:32.429087 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:20:32.429087 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:20:32.429087 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:20:32.430437 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:20:32.430437 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:20:32.430437 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:20:32.430437 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:20:32.430437 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:20:32.430437 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:20:32.430437 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:20:32.430437 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:20:32.430437 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:20:32.430437 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:20:32.430437 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:20:32.430437 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:20:32.430437 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:20:32.430437 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:20:32.430437 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:20:32.430437 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:20:32.431516 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:20:32.431516 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:20:32.431516 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:20:32.431516 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:20:32.431516 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:20:32.431516 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:20:32.431516 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:20:32.431516 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:20:32.431516 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:20:32.431516 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:20:32.431516 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 03:20:32.431516 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:20:32.432864 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf08c0-ef6c-4ba3-b86e-f6df838851fc None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:32.432864 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf08c0-ef6c-4ba3-b86e-f6df838851fc None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:32.432864 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bf08c0-ef6c-4ba3-b86e-f6df838851fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:32.433050 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 51/103] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Nov 2 03:20:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:20:32.436647 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf3e55-d62a-438a-bf97-0d66ad88ed73 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:32.453462 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-e8bf08c0-ef6c-4ba3-b86e-f6df838851fc None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:20:32.454719 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e8bf08c0-ef6c-4ba3-b86e-f6df838851fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:20:32.542653 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:32.559992 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 53/104] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:20:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 03:20:32.559992 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bf08c0-ef6c-4ba3-b86e-f6df838851fc None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:32.560358 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbf3e55-d62a-438a-bf97-0d66ad88ed73 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:32.572712 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:32.592171 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a515840b-aaef-4a2d-bc56-23997a92bc2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:32.592975 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a515840b-aaef-4a2d-bc56-23997a92bc2f None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:32.593376 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a515840b-aaef-4a2d-bc56-23997a92bc2f None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:32.593745 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a515840b-aaef-4a2d-bc56-23997a92bc2f None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:32.594494 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a515840b-aaef-4a2d-bc56-23997a92bc2f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:20:32.594870 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a515840b-aaef-4a2d-bc56-23997a92bc2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:32.596369 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a515840b-aaef-4a2d-bc56-23997a92bc2f None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:32.608143 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 52/105] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Nov 2 03:20:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:32.609008 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a515840b-aaef-4a2d-bc56-23997a92bc2f None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:32.691094 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bf08c0-ef6c-4ba3-b86e-f6df838851fc None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:32.720413 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a515840b-aaef-4a2d-bc56-23997a92bc2f None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:32.729719 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cb14c6-8a9f-4a99-8671-fc57b9bb758c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:32.730370 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cb14c6-8a9f-4a99-8671-fc57b9bb758c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:32.730654 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cb14c6-8a9f-4a99-8671-fc57b9bb758c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:32.730930 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cb14c6-8a9f-4a99-8671-fc57b9bb758c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:32.741266 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0cb14c6-8a9f-4a99-8671-fc57b9bb758c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:20:32.746637 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-c0cb14c6-8a9f-4a99-8671-fc57b9bb758c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 03:20:32.746637 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:20:32.746637 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:20:32.746637 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:20:32.746637 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:20:32.746637 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:20:32.746637 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:20:32.746637 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:20:32.746637 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:20:32.746637 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:20:32.746637 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:20:32.746637 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:20:32.746637 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:20:32.746637 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:20:32.746637 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:20:32.746637 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:20:32.747470 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:20:32.747470 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:20:32.747470 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:20:32.747470 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:20:32.747470 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:20:32.747470 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:20:32.747470 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:20:32.747470 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:20:32.747470 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:20:32.747470 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:20:32.747470 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:20:32.747470 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:20:32.747470 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:20:32.747470 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:20:32.747470 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:20:32.747470 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:20:32.748667 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:20:32.748667 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:20:32.748667 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:20:32.748667 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:20:32.748667 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:20:32.748667 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:20:32.748667 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:20:32.748667 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:20:32.748667 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:20:32.748667 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:20:32.748667 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 03:20:32.748667 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:20:32.748667 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 54/106] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Nov 2 03:20:32 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:20:32.749523 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a1ef55-829c-4add-9e75-2483ea050cee None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:32.749619 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a1ef55-829c-4add-9e75-2483ea050cee None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:32.749881 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a1ef55-829c-4add-9e75-2483ea050cee None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:32.755210 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 55/107] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:20:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:20:32.755210 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a1ef55-829c-4add-9e75-2483ea050cee None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:32.761088 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa16334-4d42-40a8-be86-df72da835bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:32.761707 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa16334-4d42-40a8-be86-df72da835bd3 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:32.761991 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa16334-4d42-40a8-be86-df72da835bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:32.762256 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa16334-4d42-40a8-be86-df72da835bd3 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:32.762915 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa16334-4d42-40a8-be86-df72da835bd3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:20:32.763220 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa16334-4d42-40a8-be86-df72da835bd3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:32.764634 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa16334-4d42-40a8-be86-df72da835bd3 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:32.772730 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 53/108] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Nov 2 03:20:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:32.773556 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa16334-4d42-40a8-be86-df72da835bd3 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:32.816621 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:32.823343 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:32.871229 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a1ef55-829c-4add-9e75-2483ea050cee None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:32.875551 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa16334-4d42-40a8-be86-df72da835bd3 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:32.904765 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9e4f51-d297-48c2-a28d-2efcc37046af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:32.905399 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e4f51-d297-48c2-a28d-2efcc37046af None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:32.905699 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e4f51-d297-48c2-a28d-2efcc37046af None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:32.905975 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9e4f51-d297-48c2-a28d-2efcc37046af None admin] PATH_INFO: `/v3/users/cc903821fbc442d9b4a30f92f10f1b17` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:32.907083 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9e4f51-d297-48c2-a28d-2efcc37046af None admin] RBAC: Authorizing `identity:get_user(user_id=cc903821fbc442d9b4a30f92f10f1b17)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:32.907226 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb4a2b1-5bb6-493e-8752-6ccf135e33af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:32.907823 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb4a2b1-5bb6-493e-8752-6ccf135e33af None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:32.908134 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb4a2b1-5bb6-493e-8752-6ccf135e33af None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:32.908409 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb4a2b1-5bb6-493e-8752-6ccf135e33af None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:32.908586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9e4f51-d297-48c2-a28d-2efcc37046af None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:32.909069 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb4a2b1-5bb6-493e-8752-6ccf135e33af None admin] RBAC: Adding query filter params (name=reader) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:20:32.909371 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb4a2b1-5bb6-493e-8752-6ccf135e33af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:32.910320 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 56/109] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Nov 2 03:20:32 2022] GET /identity/v3/users/cc903821fbc442d9b4a30f92f10f1b17 => generated 247 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:32.910731 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb4a2b1-5bb6-493e-8752-6ccf135e33af None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:32.911391 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d8d6e3-10ee-4b09-92b0-bd593f8311c0 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:32.911684 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d8d6e3-10ee-4b09-92b0-bd593f8311c0 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:32.911972 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d8d6e3-10ee-4b09-92b0-bd593f8311c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:32.922426 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 54/110] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Nov 2 03:20:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:32.923093 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb4a2b1-5bb6-493e-8752-6ccf135e33af None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:32.932874 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-89d8d6e3-10ee-4b09-92b0-bd593f8311c0 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:20:32.934070 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-89d8d6e3-10ee-4b09-92b0-bd593f8311c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:20:32.999242 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 57/111] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:20:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 03:20:33.000096 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d8d6e3-10ee-4b09-92b0-bd593f8311c0 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:33.022689 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb4a2b1-5bb6-493e-8752-6ccf135e33af None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:33.026561 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:33.059511 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64017545-9d08-4b65-949c-0534546123d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:33.060171 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64017545-9d08-4b65-949c-0534546123d5 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:33.060451 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64017545-9d08-4b65-949c-0534546123d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:33.060722 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64017545-9d08-4b65-949c-0534546123d5 None admin] PATH_INFO: `/v3/users/80c0c58af04e42a6b66aa1ea61117907` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:33.061842 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64017545-9d08-4b65-949c-0534546123d5 None admin] RBAC: Authorizing `identity:get_user(user_id=80c0c58af04e42a6b66aa1ea61117907)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:33.063289 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64017545-9d08-4b65-949c-0534546123d5 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:33.069575 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 55/112] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Nov 2 03:20:32 2022] GET /identity/v3/users/80c0c58af04e42a6b66aa1ea61117907 => generated 275 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:33.069575 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64017545-9d08-4b65-949c-0534546123d5 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:33.073570 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:33.118396 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d8d6e3-10ee-4b09-92b0-bd593f8311c0 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:33.155706 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c6dd37-edd9-4c1b-a87d-773bc18e2974 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:33.156450 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c6dd37-edd9-4c1b-a87d-773bc18e2974 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:33.156800 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c6dd37-edd9-4c1b-a87d-773bc18e2974 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:33.157364 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c6dd37-edd9-4c1b-a87d-773bc18e2974 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:33.167832 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c6dd37-edd9-4c1b-a87d-773bc18e2974 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:20:33.169174 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64017545-9d08-4b65-949c-0534546123d5 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:33.172874 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-d2c6dd37-edd9-4c1b-a87d-773bc18e2974 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 03:20:33.172874 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:20:33.172874 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:20:33.172874 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:20:33.172874 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:20:33.172874 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:20:33.172874 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:20:33.172874 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:20:33.172874 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:20:33.172874 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:20:33.172874 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:20:33.172874 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:20:33.172874 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:20:33.172874 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:20:33.172874 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:20:33.172874 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:20:33.173727 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:20:33.173727 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:20:33.173727 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:20:33.173727 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:20:33.173727 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:20:33.173727 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:20:33.173727 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:20:33.173727 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:20:33.173727 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:20:33.173727 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:20:33.173727 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:20:33.173727 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:20:33.173727 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:20:33.173727 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:20:33.173727 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:20:33.173727 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:20:33.174493 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:20:33.174493 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:20:33.174493 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:20:33.174493 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:20:33.174493 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:20:33.174493 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:20:33.174493 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:20:33.174493 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:20:33.174493 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:20:33.174493 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:20:33.174493 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 03:20:33.174493 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:20:33.174493 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 58/113] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Nov 2 03:20:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:20:33.175117 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c6dd37-edd9-4c1b-a87d-773bc18e2974 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:33.209649 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec7b84d-7109-4035-bbdb-8728f375faaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:33.210267 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec7b84d-7109-4035-bbdb-8728f375faaf None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:33.210553 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec7b84d-7109-4035-bbdb-8728f375faaf None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:33.210831 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec7b84d-7109-4035-bbdb-8728f375faaf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:33.254896 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:33.284188 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec7b84d-7109-4035-bbdb-8728f375faaf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:33.284353 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c6dd37-edd9-4c1b-a87d-773bc18e2974 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:33.285718 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec7b84d-7109-4035-bbdb-8728f375faaf None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:33.301404 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:33.318495 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcab0a25-0c83-40e0-9b98-7621097763e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:33.319129 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab0a25-0c83-40e0-9b98-7621097763e8 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:33.319411 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab0a25-0c83-40e0-9b98-7621097763e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:33.319732 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcab0a25-0c83-40e0-9b98-7621097763e8 None admin] PATH_INFO: `/v3/users/ae210bedec404fc8bd0a456ed94e7e47` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:33.334587 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcab0a25-0c83-40e0-9b98-7621097763e8 None admin] RBAC: Authorizing `identity:get_user(user_id=ae210bedec404fc8bd0a456ed94e7e47)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:33.336587 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcab0a25-0c83-40e0-9b98-7621097763e8 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:33.338178 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 59/114] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Nov 2 03:20:33 2022] GET /identity/v3/users/ae210bedec404fc8bd0a456ed94e7e47 => generated 289 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:33.338943 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcab0a25-0c83-40e0-9b98-7621097763e8 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:33.389070 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 56/115] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Nov 2 03:20:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 03:20:33.389900 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec7b84d-7109-4035-bbdb-8728f375faaf None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:33.439780 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcab0a25-0c83-40e0-9b98-7621097763e8 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:33.456916 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:33.473310 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ed1bf3-c7c4-492d-9e73-c884529a82cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:33.473917 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed1bf3-c7c4-492d-9e73-c884529a82cc None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:33.474199 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed1bf3-c7c4-492d-9e73-c884529a82cc None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:33.474472 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ed1bf3-c7c4-492d-9e73-c884529a82cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:33.486106 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec7b84d-7109-4035-bbdb-8728f375faaf None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:33.518430 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:33.519737 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cfc69e-6dd5-49f8-bd5a-66c0b688c534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:33.520399 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cfc69e-6dd5-49f8-bd5a-66c0b688c534 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:33.520680 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cfc69e-6dd5-49f8-bd5a-66c0b688c534 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:33.520951 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41cfc69e-6dd5-49f8-bd5a-66c0b688c534 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:33.521609 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cfc69e-6dd5-49f8-bd5a-66c0b688c534 None admin] RBAC: Adding query filter params (name=member) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:20:33.521911 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cfc69e-6dd5-49f8-bd5a-66c0b688c534 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:33.523274 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41cfc69e-6dd5-49f8-bd5a-66c0b688c534 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:33.532969 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 57/116] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Nov 2 03:20:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:33.533758 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cfc69e-6dd5-49f8-bd5a-66c0b688c534 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:33.544149 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ed1bf3-c7c4-492d-9e73-c884529a82cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:33.545909 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ed1bf3-c7c4-492d-9e73-c884529a82cc None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:33.632203 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41cfc69e-6dd5-49f8-bd5a-66c0b688c534 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:33.644777 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 60/117] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Nov 2 03:20:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 03:20:33.645516 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ed1bf3-c7c4-492d-9e73-c884529a82cc None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:33.665921 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a3841-05e8-4f36-adde-115491826bdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:33.666566 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a3841-05e8-4f36-adde-115491826bdb None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:33.666837 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a3841-05e8-4f36-adde-115491826bdb None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:33.667113 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a3841-05e8-4f36-adde-115491826bdb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:33.668256 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a3841-05e8-4f36-adde-115491826bdb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:33.669704 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a3841-05e8-4f36-adde-115491826bdb None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:33.671266 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 58/118] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:20:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:33.672042 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a3841-05e8-4f36-adde-115491826bdb None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:33.674432 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:33.726042 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:33.750033 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ed1bf3-c7c4-492d-9e73-c884529a82cc None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:33.770440 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a3841-05e8-4f36-adde-115491826bdb None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:33.784338 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570bd2b0-b245-451e-8311-935b15791f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:33.784966 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570bd2b0-b245-451e-8311-935b15791f39 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:33.785246 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570bd2b0-b245-451e-8311-935b15791f39 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:33.785520 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570bd2b0-b245-451e-8311-935b15791f39 None admin] PATH_INFO: `/v3/projects/958251514b4b46ef908d92c563878b13` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:33.786633 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570bd2b0-b245-451e-8311-935b15791f39 None admin] RBAC: Authorizing `identity:get_project(project_id=958251514b4b46ef908d92c563878b13)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:33.788108 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570bd2b0-b245-451e-8311-935b15791f39 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:33.789783 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 61/119] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Nov 2 03:20:33 2022] GET /identity/v3/projects/958251514b4b46ef908d92c563878b13 => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:33.792816 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570bd2b0-b245-451e-8311-935b15791f39 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:33.803534 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622e57f1-20f0-4518-9ff1-d9abf939a4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:33.804223 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622e57f1-20f0-4518-9ff1-d9abf939a4eb None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:33.804506 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622e57f1-20f0-4518-9ff1-d9abf939a4eb None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:33.804812 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622e57f1-20f0-4518-9ff1-d9abf939a4eb None admin] PATH_INFO: `/v3/projects/958251514b4b46ef908d92c563878b13` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:33.805940 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622e57f1-20f0-4518-9ff1-d9abf939a4eb None admin] RBAC: Authorizing `identity:get_project(project_id=958251514b4b46ef908d92c563878b13)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:33.807386 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622e57f1-20f0-4518-9ff1-d9abf939a4eb None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:33.809042 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 59/120] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Nov 2 03:20:33 2022] GET /identity/v3/projects/958251514b4b46ef908d92c563878b13 => generated 298 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:33.809794 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622e57f1-20f0-4518-9ff1-d9abf939a4eb None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:33.877615 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:33.891601 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570bd2b0-b245-451e-8311-935b15791f39 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:33.909220 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622e57f1-20f0-4518-9ff1-d9abf939a4eb None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:33.924210 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7318b3a-f060-4baf-88ac-7beb06327ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:33.924842 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7318b3a-f060-4baf-88ac-7beb06327ef7 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:33.925123 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7318b3a-f060-4baf-88ac-7beb06327ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:33.925404 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7318b3a-f060-4baf-88ac-7beb06327ef7 None admin] PATH_INFO: `/v3/users/80c0c58af04e42a6b66aa1ea61117907` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:33.926532 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7318b3a-f060-4baf-88ac-7beb06327ef7 None admin] RBAC: Authorizing `identity:get_user(user_id=80c0c58af04e42a6b66aa1ea61117907)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:33.927992 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7318b3a-f060-4baf-88ac-7beb06327ef7 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:33.929557 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 62/121] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Nov 2 03:20:33 2022] GET /identity/v3/users/80c0c58af04e42a6b66aa1ea61117907 => generated 275 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:33.934670 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7318b3a-f060-4baf-88ac-7beb06327ef7 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:33.942744 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f30b3c-9424-4e05-a67c-88e4733029fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:33.943374 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f30b3c-9424-4e05-a67c-88e4733029fa None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:33.943659 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f30b3c-9424-4e05-a67c-88e4733029fa None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:33.943939 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f30b3c-9424-4e05-a67c-88e4733029fa None admin] PATH_INFO: `/v3/projects/958251514b4b46ef908d92c563878b13` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:33.945075 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f30b3c-9424-4e05-a67c-88e4733029fa None admin] RBAC: Authorizing `identity:get_project(project_id=958251514b4b46ef908d92c563878b13)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:33.946579 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f30b3c-9424-4e05-a67c-88e4733029fa None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:33.947809 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:33.948316 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 60/122] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Nov 2 03:20:33 2022] GET /identity/v3/projects/958251514b4b46ef908d92c563878b13 => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:33.949132 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f30b3c-9424-4e05-a67c-88e4733029fa None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:34.039825 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7318b3a-f060-4baf-88ac-7beb06327ef7 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:34.046741 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f30b3c-9424-4e05-a67c-88e4733029fa None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:34.074716 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e02686d-2065-4f81-95ed-fc8f77726b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:34.075320 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e02686d-2065-4f81-95ed-fc8f77726b8f None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:34.075603 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e02686d-2065-4f81-95ed-fc8f77726b8f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:34.075886 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e02686d-2065-4f81-95ed-fc8f77726b8f None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:34.076636 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e02686d-2065-4f81-95ed-fc8f77726b8f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:34.078072 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e02686d-2065-4f81-95ed-fc8f77726b8f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:34.079099 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a804e08-cd84-4db9-967d-b19c4068075c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:34.079748 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a804e08-cd84-4db9-967d-b19c4068075c None admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:34.080063 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a804e08-cd84-4db9-967d-b19c4068075c None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:34.080346 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a804e08-cd84-4db9-967d-b19c4068075c None admin] PATH_INFO: `/v3/projects/958251514b4b46ef908d92c563878b13/users/ae210bedec404fc8bd0a456ed94e7e47/roles/1966cb5a13b8421aad8823a14385cb43` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:34.082234 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a804e08-cd84-4db9-967d-b19c4068075c None admin] RBAC: Authorizing `identity:create_grant(project_id=958251514b4b46ef908d92c563878b13, user_id=ae210bedec404fc8bd0a456ed94e7e47, role_id=1966cb5a13b8421aad8823a14385cb43)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:34.083680 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a804e08-cd84-4db9-967d-b19c4068075c None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:34.093022 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:34.098544 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 61/123] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Nov 2 03:20:33 2022] PUT /identity/v3/projects/958251514b4b46ef908d92c563878b13/users/ae210bedec404fc8bd0a456ed94e7e47/roles/1966cb5a13b8421aad8823a14385cb43 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:20:34.100310 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a804e08-cd84-4db9-967d-b19c4068075c None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:34.105238 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 63/124] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Nov 2 03:20:33 2022] POST /identity/v3/projects => generated 302 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:20:34.106207 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e02686d-2065-4f81-95ed-fc8f77726b8f None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:34.204238 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:34.257010 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e02686d-2065-4f81-95ed-fc8f77726b8f None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:34.274399 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a804e08-cd84-4db9-967d-b19c4068075c None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:34.300705 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da16624-9045-4dea-980d-b055404179ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:34.301497 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da16624-9045-4dea-980d-b055404179ce None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:34.301912 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da16624-9045-4dea-980d-b055404179ce None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:34.302200 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da16624-9045-4dea-980d-b055404179ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:34.316745 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6fd05d-d7ee-4690-857a-38f24b0719e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:34.317371 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6fd05d-d7ee-4690-857a-38f24b0719e2 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:34.317656 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6fd05d-d7ee-4690-857a-38f24b0719e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:34.317975 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6fd05d-d7ee-4690-857a-38f24b0719e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:34.318698 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6fd05d-d7ee-4690-857a-38f24b0719e2 None admin] RBAC: Adding query filter params (role.id=addc0db6b4ac4362b07fe21e14f89180, scope.project.id=958251514b4b46ef908d92c563878b13, user.id=cc903821fbc442d9b4a30f92f10f1b17) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:20:34.319008 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6fd05d-d7ee-4690-857a-38f24b0719e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:34.321037 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6fd05d-d7ee-4690-857a-38f24b0719e2 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:34.327454 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 62/125] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Nov 2 03:20:34 2022] GET /identity/v3/role_assignments?user.id=cc903821fbc442d9b4a30f92f10f1b17&scope.project.id=958251514b4b46ef908d92c563878b13&role.id=addc0db6b4ac4362b07fe21e14f89180 => generated 608 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:34.328291 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6fd05d-d7ee-4690-857a-38f24b0719e2 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:34.408540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da16624-9045-4dea-980d-b055404179ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:34.411238 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da16624-9045-4dea-980d-b055404179ce None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:34.417898 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:34.480096 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6fd05d-d7ee-4690-857a-38f24b0719e2 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:34.498830 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:34.536619 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db0414f-aa44-4485-b529-65f9b5d8200e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:34.537335 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0414f-aa44-4485-b529-65f9b5d8200e None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:34.537686 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0414f-aa44-4485-b529-65f9b5d8200e None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:34.538034 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0414f-aa44-4485-b529-65f9b5d8200e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:34.539685 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db0414f-aa44-4485-b529-65f9b5d8200e None admin] RBAC: Adding query filter params (role.id=f25d67f31fee4a92b3060e45a3fbe379, scope.project.id=958251514b4b46ef908d92c563878b13, user.id=80c0c58af04e42a6b66aa1ea61117907) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:20:34.540129 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db0414f-aa44-4485-b529-65f9b5d8200e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:34.542418 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db0414f-aa44-4485-b529-65f9b5d8200e None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:34.550720 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 63/126] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Nov 2 03:20:34 2022] GET /identity/v3/role_assignments?user.id=80c0c58af04e42a6b66aa1ea61117907&scope.project.id=958251514b4b46ef908d92c563878b13&role.id=f25d67f31fee4a92b3060e45a3fbe379 => generated 608 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:34.561324 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 64/127] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Nov 2 03:20:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 03:20:34.567187 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db0414f-aa44-4485-b529-65f9b5d8200e None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:34.701978 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db0414f-aa44-4485-b529-65f9b5d8200e None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:34.731217 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:34.741599 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6ca7ef-67cd-4f10-bb6e-d31cc8d36034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:34.743037 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ca7ef-67cd-4f10-bb6e-d31cc8d36034 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:34.743485 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ca7ef-67cd-4f10-bb6e-d31cc8d36034 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:34.743898 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6ca7ef-67cd-4f10-bb6e-d31cc8d36034 None admin] PATH_INFO: `/v3/projects/e0ec9f4137124973b29cd93847af3932` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:34.745267 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6ca7ef-67cd-4f10-bb6e-d31cc8d36034 None admin] RBAC: Authorizing `identity:get_project(project_id=e0ec9f4137124973b29cd93847af3932)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:34.746978 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6ca7ef-67cd-4f10-bb6e-d31cc8d36034 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:34.751104 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 64/128] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Nov 2 03:20:34 2022] GET /identity/v3/projects/e0ec9f4137124973b29cd93847af3932 => generated 312 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:34.757270 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da16624-9045-4dea-980d-b055404179ce None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:34.876540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da16624-9045-4dea-980d-b055404179ce None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:34.911648 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37090b0-02cf-4531-8057-5fe89cfbbf58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:34.912293 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37090b0-02cf-4531-8057-5fe89cfbbf58 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:34.912575 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37090b0-02cf-4531-8057-5fe89cfbbf58 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:34.912848 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37090b0-02cf-4531-8057-5fe89cfbbf58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:34.913579 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37090b0-02cf-4531-8057-5fe89cfbbf58 None admin] RBAC: Adding query filter params (role.id=7a6df2ce66694822bce7983f7d988da5, scope.project.id=e0ec9f4137124973b29cd93847af3932, user.id=80c0c58af04e42a6b66aa1ea61117907) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:20:34.913887 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37090b0-02cf-4531-8057-5fe89cfbbf58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:34.915901 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37090b0-02cf-4531-8057-5fe89cfbbf58 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:34.922034 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 65/129] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Nov 2 03:20:34 2022] GET /identity/v3/role_assignments?user.id=80c0c58af04e42a6b66aa1ea61117907&scope.project.id=e0ec9f4137124973b29cd93847af3932&role.id=7a6df2ce66694822bce7983f7d988da5 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:20:36.301726 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9226be1f-d77c-47c5-b93f-5a118f423a22 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:36.302339 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9226be1f-d77c-47c5-b93f-5a118f423a22 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:36.302339 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9226be1f-d77c-47c5-b93f-5a118f423a22 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:36.303604 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 65/130] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:20:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:20:36.308914 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f926dd-9e41-44f4-acd3-8c84cf7868fc None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:36.309343 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f926dd-9e41-44f4-acd3-8c84cf7868fc None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:36.309469 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f926dd-9e41-44f4-acd3-8c84cf7868fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:36.327057 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d6f926dd-9e41-44f4-acd3-8c84cf7868fc None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:20:36.328248 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d6f926dd-9e41-44f4-acd3-8c84cf7868fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:20:36.350669 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:36.394257 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 66/131] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:20:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 03:20:36.516588 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732cd52b-c214-41c3-94bf-ca411e27adc1 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:36.516872 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732cd52b-c214-41c3-94bf-ca411e27adc1 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:36.517149 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732cd52b-c214-41c3-94bf-ca411e27adc1 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:36.518514 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 66/132] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:20:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:20:36.523674 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cbeeda-8ea9-46c7-ac23-a90a95d73179 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:36.524083 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cbeeda-8ea9-46c7-ac23-a90a95d73179 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:36.524367 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cbeeda-8ea9-46c7-ac23-a90a95d73179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:36.542891 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-36cbeeda-8ea9-46c7-ac23-a90a95d73179 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:20:36.544122 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-36cbeeda-8ea9-46c7-ac23-a90a95d73179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:20:36.607536 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 67/133] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:20:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 03:20:36.659352 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732cd52b-c214-41c3-94bf-ca411e27adc1 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:36.759202 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732cd52b-c214-41c3-94bf-ca411e27adc1 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:36.792098 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2ae4dc-173b-4c76-a620-9f2a8ac98212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:36.792726 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ae4dc-173b-4c76-a620-9f2a8ac98212 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:36.793011 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ae4dc-173b-4c76-a620-9f2a8ac98212 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:36.793289 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2ae4dc-173b-4c76-a620-9f2a8ac98212 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:36.803165 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2ae4dc-173b-4c76-a620-9f2a8ac98212 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:20:36.809086 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-9f2ae4dc-173b-4c76-a620-9f2a8ac98212 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 03:20:36.809086 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:20:36.809086 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:20:36.809086 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:20:36.809086 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:20:36.809086 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:20:36.809086 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:20:36.809086 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:20:36.809086 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:20:36.809086 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:20:36.809086 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:20:36.809086 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:20:36.809086 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:20:36.809086 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:20:36.809086 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:20:36.809086 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:20:36.810169 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:20:36.810169 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:20:36.810169 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:20:36.810169 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:20:36.810169 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:20:36.810169 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:20:36.810169 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:20:36.810169 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:20:36.810169 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:20:36.810169 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:20:36.810169 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:20:36.810169 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:20:36.810169 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:20:36.810169 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:20:36.810169 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:20:36.810169 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:20:36.811181 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:20:36.811181 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:20:36.811181 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:20:36.811181 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:20:36.811181 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:20:36.811181 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:20:36.811181 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:20:36.811181 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:20:36.811181 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:20:36.811181 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:20:36.811181 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 03:20:36.811181 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:20:36.811181 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 67/134] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Nov 2 03:20:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:20:36.817972 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cbeeda-8ea9-46c7-ac23-a90a95d73179 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:36.875612 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2ae4dc-173b-4c76-a620-9f2a8ac98212 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:36.902410 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:36.914967 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cbeeda-8ea9-46c7-ac23-a90a95d73179 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:36.932030 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:36.948066 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88228fc1-bbb5-4eb5-a6e2-695630700639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:36.948680 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88228fc1-bbb5-4eb5-a6e2-695630700639 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:36.948963 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88228fc1-bbb5-4eb5-a6e2-695630700639 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:36.949240 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88228fc1-bbb5-4eb5-a6e2-695630700639 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:36.949899 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88228fc1-bbb5-4eb5-a6e2-695630700639 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:20:36.950202 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88228fc1-bbb5-4eb5-a6e2-695630700639 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:36.951582 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88228fc1-bbb5-4eb5-a6e2-695630700639 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:36.959626 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 68/135] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Nov 2 03:20:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:36.965806 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88228fc1-bbb5-4eb5-a6e2-695630700639 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:36.977039 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2ae4dc-173b-4c76-a620-9f2a8ac98212 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:37.010798 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a057bf5-5bae-42e9-9379-e9a6d14b3abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:37.011429 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a057bf5-5bae-42e9-9379-e9a6d14b3abc None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:37.011710 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a057bf5-5bae-42e9-9379-e9a6d14b3abc None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:37.012009 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a057bf5-5bae-42e9-9379-e9a6d14b3abc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:37.013125 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a057bf5-5bae-42e9-9379-e9a6d14b3abc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:37.014551 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a057bf5-5bae-42e9-9379-e9a6d14b3abc None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:37.016133 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 68/136] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:20:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:20:37.021383 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a057bf5-5bae-42e9-9379-e9a6d14b3abc None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:37.068491 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88228fc1-bbb5-4eb5-a6e2-695630700639 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:37.102043 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c0bd88-47ec-4a1b-b1a9-6c917ae9d405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:37.102673 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c0bd88-47ec-4a1b-b1a9-6c917ae9d405 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:37.102951 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c0bd88-47ec-4a1b-b1a9-6c917ae9d405 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:37.103224 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c0bd88-47ec-4a1b-b1a9-6c917ae9d405 None admin] PATH_INFO: `/v3/users/ae210bedec404fc8bd0a456ed94e7e47` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:37.104406 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c0bd88-47ec-4a1b-b1a9-6c917ae9d405 None admin] RBAC: Authorizing `identity:get_user(user_id=ae210bedec404fc8bd0a456ed94e7e47)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:37.105837 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c0bd88-47ec-4a1b-b1a9-6c917ae9d405 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:37.107392 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 69/137] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Nov 2 03:20:36 2022] GET /identity/v3/users/ae210bedec404fc8bd0a456ed94e7e47 => generated 289 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:20:37.112952 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c0bd88-47ec-4a1b-b1a9-6c917ae9d405 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:37.118848 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a057bf5-5bae-42e9-9379-e9a6d14b3abc None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:37.131578 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:37.141340 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:37.152920 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c3b9e7-8729-45bf-8f3a-870f78382615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:37.153532 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c3b9e7-8729-45bf-8f3a-870f78382615 None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:37.153826 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c3b9e7-8729-45bf-8f3a-870f78382615 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:37.154104 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c3b9e7-8729-45bf-8f3a-870f78382615 None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:37.154752 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c3b9e7-8729-45bf-8f3a-870f78382615 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:37.156196 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c3b9e7-8729-45bf-8f3a-870f78382615 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:37.176786 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 69/138] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Nov 2 03:20:37 2022] POST /identity/v3/users => generated 283 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 03:20:37.218563 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c0bd88-47ec-4a1b-b1a9-6c917ae9d405 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:37.254316 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb941d6-850f-46b2-89da-7a21935ec67d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:37.254942 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb941d6-850f-46b2-89da-7a21935ec67d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:37.255221 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb941d6-850f-46b2-89da-7a21935ec67d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:37.255495 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb941d6-850f-46b2-89da-7a21935ec67d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:37.326577 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb941d6-850f-46b2-89da-7a21935ec67d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:37.328104 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb941d6-850f-46b2-89da-7a21935ec67d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:37.346234 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:37.427793 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 70/139] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Nov 2 03:20:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Nov 02 03:20:37.433614 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c3b9e7-8729-45bf-8f3a-870f78382615 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:37.548975 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c3b9e7-8729-45bf-8f3a-870f78382615 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:37.587698 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87cf477-b64d-4c93-8bad-801f4d69325d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:37.588378 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87cf477-b64d-4c93-8bad-801f4d69325d None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:37.588656 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87cf477-b64d-4c93-8bad-801f4d69325d None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:37.588926 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87cf477-b64d-4c93-8bad-801f4d69325d None admin] PATH_INFO: `/v3/projects/958251514b4b46ef908d92c563878b13` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:37.590047 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87cf477-b64d-4c93-8bad-801f4d69325d None admin] RBAC: Authorizing `identity:get_project(project_id=958251514b4b46ef908d92c563878b13)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:37.591475 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87cf477-b64d-4c93-8bad-801f4d69325d None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:37.593514 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 70/140] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Nov 2 03:20:37 2022] GET /identity/v3/projects/958251514b4b46ef908d92c563878b13 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:37.599480 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb941d6-850f-46b2-89da-7a21935ec67d None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:37.709768 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb941d6-850f-46b2-89da-7a21935ec67d None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:37.734670 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:37.752985 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fef905-8be9-49b8-b750-9b5523f6d43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:37.753669 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fef905-8be9-49b8-b750-9b5523f6d43f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:37.753951 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fef905-8be9-49b8-b750-9b5523f6d43f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:37.754230 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fef905-8be9-49b8-b750-9b5523f6d43f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:37.755007 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fef905-8be9-49b8-b750-9b5523f6d43f None admin] RBAC: Adding query filter params (role.id=1966cb5a13b8421aad8823a14385cb43, scope.project.id=958251514b4b46ef908d92c563878b13, user.id=ae210bedec404fc8bd0a456ed94e7e47) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:20:37.755320 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fef905-8be9-49b8-b750-9b5523f6d43f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:37.756883 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fef905-8be9-49b8-b750-9b5523f6d43f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:37.763500 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 71/141] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Nov 2 03:20:37 2022] GET /identity/v3/role_assignments?user.id=ae210bedec404fc8bd0a456ed94e7e47&scope.project.id=958251514b4b46ef908d92c563878b13&role.id=1966cb5a13b8421aad8823a14385cb43 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:39.428974 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59284dc8-6462-43cb-a04b-0016557cbed2 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:39.429484 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59284dc8-6462-43cb-a04b-0016557cbed2 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:39.429484 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59284dc8-6462-43cb-a04b-0016557cbed2 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:39.431525 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 71/142] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:20:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:20:39.436441 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c85aa4f-02b3-42b1-a9b8-3df45f02c172 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:39.436841 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c85aa4f-02b3-42b1-a9b8-3df45f02c172 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:39.437014 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c85aa4f-02b3-42b1-a9b8-3df45f02c172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:39.455116 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-5c85aa4f-02b3-42b1-a9b8-3df45f02c172 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:20:39.456350 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5c85aa4f-02b3-42b1-a9b8-3df45f02c172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:20:39.537283 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 72/143] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:20:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 03:20:39.607475 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb8180-e003-4c10-aafc-f44e1049345e None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:39.607768 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb8180-e003-4c10-aafc-f44e1049345e None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:39.608077 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb8180-e003-4c10-aafc-f44e1049345e None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:39.609574 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 72/144] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:20:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:20:39.614772 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af46254d-bede-4de1-9f05-c0b6fd1b7b9f None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:39.615068 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af46254d-bede-4de1-9f05-c0b6fd1b7b9f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:39.615345 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af46254d-bede-4de1-9f05-c0b6fd1b7b9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:39.633819 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-af46254d-bede-4de1-9f05-c0b6fd1b7b9f None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:20:39.635005 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-af46254d-bede-4de1-9f05-c0b6fd1b7b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:20:39.673232 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27685507-3838-4292-9afe-b79c8c8e7bd5 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:39.673531 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27685507-3838-4292-9afe-b79c8c8e7bd5 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:39.673790 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27685507-3838-4292-9afe-b79c8c8e7bd5 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:39.675139 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 73/145] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:20:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 03:20:39.680243 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cf2363-bdea-4b6e-96f0-a2596e2a2950 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:39.680477 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cf2363-bdea-4b6e-96f0-a2596e2a2950 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:39.680748 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64cf2363-bdea-4b6e-96f0-a2596e2a2950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:39.698875 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-64cf2363-bdea-4b6e-96f0-a2596e2a2950 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:20:39.700095 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-64cf2363-bdea-4b6e-96f0-a2596e2a2950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:20:39.706657 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 73/146] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:20:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 03:20:39.743020 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:39.770125 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 74/147] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:20:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 03:20:39.799858 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af46254d-bede-4de1-9f05-c0b6fd1b7b9f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:39.842380 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:39.898281 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af46254d-bede-4de1-9f05-c0b6fd1b7b9f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:39.931941 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10af3cce-c1fb-45e8-a142-61bfa72d3483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:39.932540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10af3cce-c1fb-45e8-a142-61bfa72d3483 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:39.932820 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10af3cce-c1fb-45e8-a142-61bfa72d3483 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:39.933091 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10af3cce-c1fb-45e8-a142-61bfa72d3483 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:39.942877 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10af3cce-c1fb-45e8-a142-61bfa72d3483 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:20:39.947742 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-10af3cce-c1fb-45e8-a142-61bfa72d3483 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 03:20:39.947742 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:20:39.947742 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:20:39.947742 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:20:39.947742 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:20:39.947742 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:20:39.947742 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:20:39.947742 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:20:39.947742 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:20:39.947742 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:20:39.947742 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:20:39.947742 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:20:39.947742 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:20:39.947742 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:20:39.947742 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:20:39.947742 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:20:39.948614 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:20:39.948614 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:20:39.948614 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:20:39.948614 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:20:39.948614 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:20:39.948614 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:20:39.948614 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:20:39.948614 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:20:39.948614 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:20:39.948614 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:20:39.948614 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:20:39.948614 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:20:39.948614 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:20:39.948614 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:20:39.948614 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:20:39.948614 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:20:39.949554 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:20:39.949554 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:20:39.949554 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:20:39.949554 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:20:39.949554 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:20:39.949554 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:20:39.949554 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:20:39.949554 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:20:39.949554 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:20:39.949554 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:20:39.949554 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 03:20:39.949554 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:20:39.949554 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 74/148] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Nov 2 03:20:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:20:39.956099 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cf2363-bdea-4b6e-96f0-a2596e2a2950 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:39.979470 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10af3cce-c1fb-45e8-a142-61bfa72d3483 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:40.056984 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64cf2363-bdea-4b6e-96f0-a2596e2a2950 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:40.085125 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10af3cce-c1fb-45e8-a142-61bfa72d3483 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:40.086644 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:40.090756 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977efa9f-c1cc-463f-b678-6fceca2e28d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:40.091366 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977efa9f-c1cc-463f-b678-6fceca2e28d3 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:40.091647 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977efa9f-c1cc-463f-b678-6fceca2e28d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:40.091917 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977efa9f-c1cc-463f-b678-6fceca2e28d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:40.092614 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977efa9f-c1cc-463f-b678-6fceca2e28d3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:20:40.092914 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977efa9f-c1cc-463f-b678-6fceca2e28d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:40.094285 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977efa9f-c1cc-463f-b678-6fceca2e28d3 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:40.102878 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 75/149] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Nov 2 03:20:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:40.103672 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977efa9f-c1cc-463f-b678-6fceca2e28d3 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:40.124592 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e39a114-0a1f-402d-b486-8f8134e595bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:40.125224 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e39a114-0a1f-402d-b486-8f8134e595bb None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:40.125500 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e39a114-0a1f-402d-b486-8f8134e595bb None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:40.125772 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e39a114-0a1f-402d-b486-8f8134e595bb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:40.135728 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e39a114-0a1f-402d-b486-8f8134e595bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:20:40.140539 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-0e39a114-0a1f-402d-b486-8f8134e595bb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 03:20:40.140539 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:20:40.140539 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:20:40.140539 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:20:40.140539 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:20:40.140539 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:20:40.140539 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:20:40.140539 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:20:40.140539 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:20:40.140539 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:20:40.140539 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:20:40.140539 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:20:40.140539 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:20:40.140539 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:20:40.140539 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:20:40.140539 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:20:40.141409 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:20:40.141409 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:20:40.141409 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:20:40.141409 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:20:40.141409 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:20:40.141409 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:20:40.141409 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:20:40.141409 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:20:40.141409 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:20:40.141409 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:20:40.141409 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:20:40.141409 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:20:40.141409 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:20:40.141409 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:20:40.141409 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:20:40.141409 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:20:40.142206 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:20:40.142206 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:20:40.142206 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:20:40.142206 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:20:40.142206 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:20:40.142206 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:20:40.142206 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:20:40.142206 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:20:40.142206 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:20:40.142206 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:20:40.142206 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 03:20:40.142206 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:20:40.142206 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 75/150] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Nov 2 03:20:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 03:20:40.142954 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e39a114-0a1f-402d-b486-8f8134e595bb None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:40.143542 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:40.204359 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977efa9f-c1cc-463f-b678-6fceca2e28d3 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:40.238253 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b126d96b-0f7c-44d8-9144-3d2b49430839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:40.238877 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b126d96b-0f7c-44d8-9144-3d2b49430839 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:40.239160 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b126d96b-0f7c-44d8-9144-3d2b49430839 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:40.239427 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b126d96b-0f7c-44d8-9144-3d2b49430839 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:40.240545 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b126d96b-0f7c-44d8-9144-3d2b49430839 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:40.241977 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b126d96b-0f7c-44d8-9144-3d2b49430839 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:40.243225 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e39a114-0a1f-402d-b486-8f8134e595bb None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:40.244149 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 76/151] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:20:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:40.244918 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b126d96b-0f7c-44d8-9144-3d2b49430839 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:40.277363 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39b823-68ca-40d4-b85a-e1b7097616c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:40.277992 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39b823-68ca-40d4-b85a-e1b7097616c5 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:40.278270 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39b823-68ca-40d4-b85a-e1b7097616c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:40.278540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f39b823-68ca-40d4-b85a-e1b7097616c5 None admin] PATH_INFO: `/v3/users/0cd9488c4d2344c68b423e381c5ed58c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:40.291840 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f39b823-68ca-40d4-b85a-e1b7097616c5 None admin] RBAC: Authorizing `identity:get_user(user_id=0cd9488c4d2344c68b423e381c5ed58c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:40.293340 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f39b823-68ca-40d4-b85a-e1b7097616c5 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:40.294936 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 76/152] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Nov 2 03:20:40 2022] GET /identity/v3/users/0cd9488c4d2344c68b423e381c5ed58c => generated 283 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:40.295672 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39b823-68ca-40d4-b85a-e1b7097616c5 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:40.318737 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:40.346578 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b126d96b-0f7c-44d8-9144-3d2b49430839 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:40.353767 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:40.380786 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb7dbbe-c800-47e5-9f55-dcb8d10c98b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:40.381394 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb7dbbe-c800-47e5-9f55-dcb8d10c98b5 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:40.381674 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb7dbbe-c800-47e5-9f55-dcb8d10c98b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:40.381949 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb7dbbe-c800-47e5-9f55-dcb8d10c98b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:40.382607 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb7dbbe-c800-47e5-9f55-dcb8d10c98b5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:20:40.383454 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb7dbbe-c800-47e5-9f55-dcb8d10c98b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:40.384918 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb7dbbe-c800-47e5-9f55-dcb8d10c98b5 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:40.392851 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 77/153] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Nov 2 03:20:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:40.392851 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb7dbbe-c800-47e5-9f55-dcb8d10c98b5 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:40.397642 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f39b823-68ca-40d4-b85a-e1b7097616c5 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:40.432309 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa7969f-6da6-403d-b711-18d9b33150a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:40.432918 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa7969f-6da6-403d-b711-18d9b33150a0 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:40.433197 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa7969f-6da6-403d-b711-18d9b33150a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:40.433470 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa7969f-6da6-403d-b711-18d9b33150a0 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:40.434114 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa7969f-6da6-403d-b711-18d9b33150a0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:40.435545 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa7969f-6da6-403d-b711-18d9b33150a0 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:40.461699 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 77/154] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Nov 2 03:20:40 2022] POST /identity/v3/users => generated 297 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:20:40.462508 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa7969f-6da6-403d-b711-18d9b33150a0 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:40.508689 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb7dbbe-c800-47e5-9f55-dcb8d10c98b5 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:40.555971 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc45426-9e9f-45ac-b29d-806cadd9701a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:40.556638 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc45426-9e9f-45ac-b29d-806cadd9701a None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:40.556919 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc45426-9e9f-45ac-b29d-806cadd9701a None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:40.557194 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc45426-9e9f-45ac-b29d-806cadd9701a None admin] PATH_INFO: `/v3/projects/953c751045ff4e78a8125850cf1d07fd` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:40.559402 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc45426-9e9f-45ac-b29d-806cadd9701a None admin] RBAC: Authorizing `identity:get_project(project_id=953c751045ff4e78a8125850cf1d07fd)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:40.560910 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc45426-9e9f-45ac-b29d-806cadd9701a None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:40.562782 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 78/155] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Nov 2 03:20:40 2022] GET /identity/v3/projects/953c751045ff4e78a8125850cf1d07fd => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:40.568552 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc45426-9e9f-45ac-b29d-806cadd9701a None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:40.572815 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:40.601129 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa7969f-6da6-403d-b711-18d9b33150a0 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:40.611774 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:40.644654 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f842e0ee-b35d-463f-afdf-9b5f60086217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:40.645297 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842e0ee-b35d-463f-afdf-9b5f60086217 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:40.645572 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842e0ee-b35d-463f-afdf-9b5f60086217 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:40.645842 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f842e0ee-b35d-463f-afdf-9b5f60086217 None admin] PATH_INFO: `/v3/users/0cd9488c4d2344c68b423e381c5ed58c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:40.646992 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f842e0ee-b35d-463f-afdf-9b5f60086217 None admin] RBAC: Authorizing `identity:get_user(user_id=0cd9488c4d2344c68b423e381c5ed58c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:40.648995 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f842e0ee-b35d-463f-afdf-9b5f60086217 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:40.650637 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 78/156] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Nov 2 03:20:40 2022] GET /identity/v3/users/0cd9488c4d2344c68b423e381c5ed58c => generated 283 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:40.656231 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f842e0ee-b35d-463f-afdf-9b5f60086217 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:40.693518 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc45426-9e9f-45ac-b29d-806cadd9701a None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:40.731089 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bccd1f-2551-4a96-aefd-c49b31047549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:40.731696 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bccd1f-2551-4a96-aefd-c49b31047549 None admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:40.731995 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bccd1f-2551-4a96-aefd-c49b31047549 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:40.732277 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bccd1f-2551-4a96-aefd-c49b31047549 None admin] PATH_INFO: `/v3/projects/953c751045ff4e78a8125850cf1d07fd/users/0cd9488c4d2344c68b423e381c5ed58c/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:40.734251 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bccd1f-2551-4a96-aefd-c49b31047549 None admin] RBAC: Authorizing `identity:create_grant(project_id=953c751045ff4e78a8125850cf1d07fd, user_id=0cd9488c4d2344c68b423e381c5ed58c, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:40.737436 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bccd1f-2551-4a96-aefd-c49b31047549 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:40.754088 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 79/157] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Nov 2 03:20:40 2022] PUT /identity/v3/projects/953c751045ff4e78a8125850cf1d07fd/users/0cd9488c4d2344c68b423e381c5ed58c/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 03:20:40.780715 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f842e0ee-b35d-463f-afdf-9b5f60086217 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:40.823938 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8905cb0c-6bf3-448b-b03a-e51844d2498f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:40.824624 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8905cb0c-6bf3-448b-b03a-e51844d2498f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:40.824900 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8905cb0c-6bf3-448b-b03a-e51844d2498f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:40.825167 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8905cb0c-6bf3-448b-b03a-e51844d2498f None admin] PATH_INFO: `/v3/projects/953c751045ff4e78a8125850cf1d07fd` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:40.826356 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8905cb0c-6bf3-448b-b03a-e51844d2498f None admin] RBAC: Authorizing `identity:get_project(project_id=953c751045ff4e78a8125850cf1d07fd)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:40.827817 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8905cb0c-6bf3-448b-b03a-e51844d2498f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:40.829724 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 79/158] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Nov 2 03:20:40 2022] GET /identity/v3/projects/953c751045ff4e78a8125850cf1d07fd => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:20:40.835474 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bccd1f-2551-4a96-aefd-c49b31047549 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:40.950552 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:40.959122 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bccd1f-2551-4a96-aefd-c49b31047549 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:40.995080 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f322f3-db60-41ef-9e8b-131bb62854f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:40.995677 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f322f3-db60-41ef-9e8b-131bb62854f3 None admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:40.996066 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f322f3-db60-41ef-9e8b-131bb62854f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:40.996268 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f322f3-db60-41ef-9e8b-131bb62854f3 None admin] PATH_INFO: `/v3/projects/953c751045ff4e78a8125850cf1d07fd/users/0cd9488c4d2344c68b423e381c5ed58c/roles/f25d67f31fee4a92b3060e45a3fbe379` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:40.998509 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f322f3-db60-41ef-9e8b-131bb62854f3 None admin] RBAC: Authorizing `identity:create_grant(project_id=953c751045ff4e78a8125850cf1d07fd, user_id=0cd9488c4d2344c68b423e381c5ed58c, role_id=f25d67f31fee4a92b3060e45a3fbe379)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:40.999993 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f322f3-db60-41ef-9e8b-131bb62854f3 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:41.017894 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 80/159] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Nov 2 03:20:40 2022] PUT /identity/v3/projects/953c751045ff4e78a8125850cf1d07fd/users/0cd9488c4d2344c68b423e381c5ed58c/roles/f25d67f31fee4a92b3060e45a3fbe379 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:20:42.741534 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a5c475-649a-4287-9b61-b26fc3762c03 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:42.742097 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a5c475-649a-4287-9b61-b26fc3762c03 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:42.742097 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a5c475-649a-4287-9b61-b26fc3762c03 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:42.743453 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 80/160] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:20:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:20:42.748776 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef08be4b-13f3-4df2-a5f0-9d3c0d2bf2ee None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:42.749181 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef08be4b-13f3-4df2-a5f0-9d3c0d2bf2ee None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:42.749362 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef08be4b-13f3-4df2-a5f0-9d3c0d2bf2ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:42.767994 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-ef08be4b-13f3-4df2-a5f0-9d3c0d2bf2ee None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:20:42.769209 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ef08be4b-13f3-4df2-a5f0-9d3c0d2bf2ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:20:42.844724 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 81/161] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:20:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 03:20:42.937062 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae99cc11-256e-4643-bca6-c73bb7beb5f3 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:42.937368 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae99cc11-256e-4643-bca6-c73bb7beb5f3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:42.937604 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae99cc11-256e-4643-bca6-c73bb7beb5f3 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:42.939129 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 81/162] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:20:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:20:42.944487 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98786bc8-187f-4196-b550-56e02546e5c2 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:42.944778 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98786bc8-187f-4196-b550-56e02546e5c2 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:42.945057 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98786bc8-187f-4196-b550-56e02546e5c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:42.963931 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-98786bc8-187f-4196-b550-56e02546e5c2 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:20:42.965148 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-98786bc8-187f-4196-b550-56e02546e5c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:20:42.967260 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c839f-1e36-47dd-827f-a0770b302ab6 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:42.967734 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c839f-1e36-47dd-827f-a0770b302ab6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:42.968245 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7c839f-1e36-47dd-827f-a0770b302ab6 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:42.969796 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 82/163] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:20:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 03:20:42.974889 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b4fbb-e92b-44e7-bd3b-ed8c05dcff2e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:42.975393 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b4fbb-e92b-44e7-bd3b-ed8c05dcff2e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:42.975866 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85b4fbb-e92b-44e7-bd3b-ed8c05dcff2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:42.995998 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d85b4fbb-e92b-44e7-bd3b-ed8c05dcff2e None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:20:42.997798 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d85b4fbb-e92b-44e7-bd3b-ed8c05dcff2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:20:43.007069 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:43.054450 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 82/164] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:20:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 03:20:43.081113 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 83/165] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:20:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 03:20:43.128075 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98786bc8-187f-4196-b550-56e02546e5c2 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:43.156527 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:43.223440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc48c0ea-11a5-46d8-91ad-3edce7252e8a None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:43.223736 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc48c0ea-11a5-46d8-91ad-3edce7252e8a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:43.224030 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc48c0ea-11a5-46d8-91ad-3edce7252e8a None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:43.225659 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 84/166] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:20:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:20:43.231335 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934e1d8-50e6-4ed2-8e99-bd02dd586883 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:43.231621 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934e1d8-50e6-4ed2-8e99-bd02dd586883 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:43.231897 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6934e1d8-50e6-4ed2-8e99-bd02dd586883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:43.259265 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-6934e1d8-50e6-4ed2-8e99-bd02dd586883 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:20:43.261384 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6934e1d8-50e6-4ed2-8e99-bd02dd586883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:20:43.265753 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98786bc8-187f-4196-b550-56e02546e5c2 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:43.328337 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cdb7d4-2318-4c5b-86da-0bd99f30689c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:43.329342 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cdb7d4-2318-4c5b-86da-0bd99f30689c None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:43.329818 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cdb7d4-2318-4c5b-86da-0bd99f30689c None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:43.330284 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cdb7d4-2318-4c5b-86da-0bd99f30689c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:43.351164 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cdb7d4-2318-4c5b-86da-0bd99f30689c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:20:43.356979 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-34cdb7d4-2318-4c5b-86da-0bd99f30689c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 03:20:43.356979 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:20:43.356979 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:20:43.356979 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:20:43.356979 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:20:43.356979 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:20:43.356979 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:20:43.356979 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:20:43.356979 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:20:43.356979 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:20:43.356979 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:20:43.356979 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:20:43.356979 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:20:43.356979 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:20:43.356979 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:20:43.356979 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:20:43.357835 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:20:43.357835 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:20:43.357835 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:20:43.357835 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:20:43.357835 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:20:43.357835 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:20:43.357835 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:20:43.357835 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:20:43.357835 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:20:43.357835 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:20:43.357835 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:20:43.357835 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:20:43.357835 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:20:43.357835 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:20:43.357835 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:20:43.357835 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:20:43.358760 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:20:43.358760 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:20:43.358760 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:20:43.358760 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:20:43.358760 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:20:43.358760 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:20:43.358760 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:20:43.358760 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:20:43.358760 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:20:43.358760 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:20:43.358760 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 03:20:43.358760 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:20:43.366199 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 83/167] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Nov 2 03:20:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:20:43.368801 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cdb7d4-2318-4c5b-86da-0bd99f30689c None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:43.390731 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 85/168] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:20:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 03:20:43.391494 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6934e1d8-50e6-4ed2-8e99-bd02dd586883 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:43.470491 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:43.492353 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:43.515905 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6934e1d8-50e6-4ed2-8e99-bd02dd586883 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:43.533184 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cdb7d4-2318-4c5b-86da-0bd99f30689c None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:43.567924 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc58511-5bc0-44d7-a07e-d190f827467b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:43.568984 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc58511-5bc0-44d7-a07e-d190f827467b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:43.569270 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc58511-5bc0-44d7-a07e-d190f827467b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:43.569967 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc58511-5bc0-44d7-a07e-d190f827467b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:43.571891 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc58511-5bc0-44d7-a07e-d190f827467b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:43.573594 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc58511-5bc0-44d7-a07e-d190f827467b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:43.575399 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 86/169] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:20:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:43.577083 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc58511-5bc0-44d7-a07e-d190f827467b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:43.577922 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2fc6e8-3fd4-4f85-872d-f5e8359d346b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:43.578716 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2fc6e8-3fd4-4f85-872d-f5e8359d346b None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:43.579058 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2fc6e8-3fd4-4f85-872d-f5e8359d346b None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:43.579451 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2fc6e8-3fd4-4f85-872d-f5e8359d346b None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:43.580245 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2fc6e8-3fd4-4f85-872d-f5e8359d346b None admin] RBAC: Adding query filter params (name=member) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:20:43.580548 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2fc6e8-3fd4-4f85-872d-f5e8359d346b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:43.582972 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2fc6e8-3fd4-4f85-872d-f5e8359d346b None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:43.597438 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 84/170] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Nov 2 03:20:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:43.597438 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2fc6e8-3fd4-4f85-872d-f5e8359d346b None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:43.681842 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc58511-5bc0-44d7-a07e-d190f827467b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:43.700242 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2fc6e8-3fd4-4f85-872d-f5e8359d346b None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:43.715659 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d57010-17d1-420b-bb3e-2720583d64bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:43.716332 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d57010-17d1-420b-bb3e-2720583d64bc None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:43.716616 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d57010-17d1-420b-bb3e-2720583d64bc None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:43.716893 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d57010-17d1-420b-bb3e-2720583d64bc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:43.725073 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:43.728170 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d57010-17d1-420b-bb3e-2720583d64bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:20:43.732668 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec6c3ed-01d0-4eb6-b72c-1072af102676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:43.733435 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-18d57010-17d1-420b-bb3e-2720583d64bc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 03:20:43.733435 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:20:43.733435 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:20:43.733435 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:20:43.733435 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:20:43.733435 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:20:43.733435 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:20:43.733435 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:20:43.733435 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:20:43.733435 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:20:43.733435 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:20:43.733435 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:20:43.733435 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:20:43.733435 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:20:43.733435 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:20:43.733435 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:20:43.734484 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:20:43.734484 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:20:43.734484 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:20:43.734484 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:20:43.734484 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:20:43.734484 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:20:43.734484 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:20:43.734484 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:20:43.734484 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:20:43.734484 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:20:43.734484 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:20:43.734484 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:20:43.734484 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:20:43.734484 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:20:43.734484 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:20:43.734484 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:20:43.740448 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:20:43.740448 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:20:43.740448 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:20:43.740448 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:20:43.740448 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:20:43.740448 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:20:43.740448 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:20:43.740448 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:20:43.740448 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:20:43.740448 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:20:43.740448 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 03:20:43.740448 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:20:43.740448 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 87/171] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Nov 2 03:20:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:20:43.740448 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d57010-17d1-420b-bb3e-2720583d64bc None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:43.741176 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6c3ed-01d0-4eb6-b72c-1072af102676 None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:43.741176 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6c3ed-01d0-4eb6-b72c-1072af102676 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:43.741176 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6c3ed-01d0-4eb6-b72c-1072af102676 None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:43.741362 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec6c3ed-01d0-4eb6-b72c-1072af102676 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:43.743224 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec6c3ed-01d0-4eb6-b72c-1072af102676 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:43.753264 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:43.764774 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 85/172] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Nov 2 03:20:43 2022] POST /identity/v3/users => generated 297 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:20:43.765647 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec6c3ed-01d0-4eb6-b72c-1072af102676 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:43.856293 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d57010-17d1-420b-bb3e-2720583d64bc None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:43.895317 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8992f2e8-b98c-4fc9-9f60-99f8c2126869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:43.895993 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8992f2e8-b98c-4fc9-9f60-99f8c2126869 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:43.896272 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8992f2e8-b98c-4fc9-9f60-99f8c2126869 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:43.896546 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8992f2e8-b98c-4fc9-9f60-99f8c2126869 None admin] PATH_INFO: `/v3/users/3577dce245ab44cb9c8c6ec6d3cb73bd` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:43.897807 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec6c3ed-01d0-4eb6-b72c-1072af102676 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:43.912292 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8992f2e8-b98c-4fc9-9f60-99f8c2126869 None admin] RBAC: Authorizing `identity:get_user(user_id=3577dce245ab44cb9c8c6ec6d3cb73bd)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:43.913805 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8992f2e8-b98c-4fc9-9f60-99f8c2126869 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:43.920144 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 88/173] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Nov 2 03:20:43 2022] GET /identity/v3/users/3577dce245ab44cb9c8c6ec6d3cb73bd => generated 297 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:43.920144 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8992f2e8-b98c-4fc9-9f60-99f8c2126869 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:43.943259 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3753849e-600b-414f-b7b2-f202ebe99617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:43.945052 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753849e-600b-414f-b7b2-f202ebe99617 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:43.945052 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753849e-600b-414f-b7b2-f202ebe99617 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:43.945052 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3753849e-600b-414f-b7b2-f202ebe99617 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:43.956728 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3753849e-600b-414f-b7b2-f202ebe99617 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:20:43.962650 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-3753849e-600b-414f-b7b2-f202ebe99617 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 03:20:43.962650 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:20:43.962650 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:20:43.962650 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:20:43.962650 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:20:43.962650 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:20:43.962650 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:20:43.962650 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:20:43.962650 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:20:43.962650 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:20:43.962650 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:20:43.962650 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:20:43.962650 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:20:43.962650 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:20:43.962650 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:20:43.962650 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:20:43.963641 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:20:43.963641 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:20:43.963641 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:20:43.963641 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:20:43.963641 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:20:43.963641 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:20:43.963641 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:20:43.963641 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:20:43.963641 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:20:43.963641 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:20:43.963641 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:20:43.963641 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:20:43.963641 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:20:43.963641 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:20:43.963641 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:20:43.963641 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:20:43.964557 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:20:43.964557 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:20:43.964557 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:20:43.964557 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:20:43.964557 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:20:43.964557 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:20:43.964557 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:20:43.964557 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:20:43.964557 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:20:43.964557 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:20:43.964557 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 03:20:43.964557 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:20:43.964557 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 86/174] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Nov 2 03:20:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:20:43.965209 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3753849e-600b-414f-b7b2-f202ebe99617 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:43.990846 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:44.029440 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:44.043624 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8992f2e8-b98c-4fc9-9f60-99f8c2126869 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:44.088297 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3753849e-600b-414f-b7b2-f202ebe99617 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:44.088775 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e683e7-2008-485d-9687-2477435a354a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:44.089405 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e683e7-2008-485d-9687-2477435a354a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:44.089962 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e683e7-2008-485d-9687-2477435a354a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:44.090238 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e683e7-2008-485d-9687-2477435a354a None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:44.091861 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e683e7-2008-485d-9687-2477435a354a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:20:44.092198 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e683e7-2008-485d-9687-2477435a354a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:44.093603 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e683e7-2008-485d-9687-2477435a354a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:44.104332 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 89/175] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Nov 2 03:20:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:44.105010 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e683e7-2008-485d-9687-2477435a354a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:44.126628 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45d6d71-0aa2-48c5-a347-31bb3d1713e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:44.127630 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d6d71-0aa2-48c5-a347-31bb3d1713e3 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:44.128124 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d6d71-0aa2-48c5-a347-31bb3d1713e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:44.128615 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45d6d71-0aa2-48c5-a347-31bb3d1713e3 None admin] PATH_INFO: `/v3/projects/953c751045ff4e78a8125850cf1d07fd` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:44.130816 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45d6d71-0aa2-48c5-a347-31bb3d1713e3 None admin] RBAC: Authorizing `identity:get_project(project_id=953c751045ff4e78a8125850cf1d07fd)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:44.132572 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45d6d71-0aa2-48c5-a347-31bb3d1713e3 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:44.135001 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 87/176] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Nov 2 03:20:43 2022] GET /identity/v3/projects/953c751045ff4e78a8125850cf1d07fd => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:44.135995 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45d6d71-0aa2-48c5-a347-31bb3d1713e3 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:44.230682 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e683e7-2008-485d-9687-2477435a354a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:44.242926 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45d6d71-0aa2-48c5-a347-31bb3d1713e3 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:44.252648 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:44.268525 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c68131-6e81-4573-95b5-fb31810aeee0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:44.269144 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c68131-6e81-4573-95b5-fb31810aeee0 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:44.269426 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c68131-6e81-4573-95b5-fb31810aeee0 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:44.269700 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c68131-6e81-4573-95b5-fb31810aeee0 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:44.270371 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c68131-6e81-4573-95b5-fb31810aeee0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:20:44.270674 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c68131-6e81-4573-95b5-fb31810aeee0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:44.272551 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c68131-6e81-4573-95b5-fb31810aeee0 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:44.278234 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:44.282942 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 90/177] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Nov 2 03:20:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:44.284113 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c68131-6e81-4573-95b5-fb31810aeee0 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:44.288429 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ec3879-44e5-47e6-8c57-235a92ec5da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:44.288764 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec3879-44e5-47e6-8c57-235a92ec5da8 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:44.289585 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec3879-44e5-47e6-8c57-235a92ec5da8 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:44.289934 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ec3879-44e5-47e6-8c57-235a92ec5da8 None admin] PATH_INFO: `/v3/users/0cd9488c4d2344c68b423e381c5ed58c` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:44.291169 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ec3879-44e5-47e6-8c57-235a92ec5da8 None admin] RBAC: Authorizing `identity:get_user(user_id=0cd9488c4d2344c68b423e381c5ed58c)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:44.292885 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ec3879-44e5-47e6-8c57-235a92ec5da8 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:44.294693 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 88/178] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Nov 2 03:20:44 2022] GET /identity/v3/users/0cd9488c4d2344c68b423e381c5ed58c => generated 283 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:44.295993 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ec3879-44e5-47e6-8c57-235a92ec5da8 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:44.401851 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c68131-6e81-4573-95b5-fb31810aeee0 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:44.410083 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ec3879-44e5-47e6-8c57-235a92ec5da8 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:44.450288 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309a4e7c-9ac8-403c-9acb-fdfc016fd3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:44.451576 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309a4e7c-9ac8-403c-9acb-fdfc016fd3bb None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:44.452045 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309a4e7c-9ac8-403c-9acb-fdfc016fd3bb None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:44.452328 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309a4e7c-9ac8-403c-9acb-fdfc016fd3bb None admin] PATH_INFO: `/v3/projects/953c751045ff4e78a8125850cf1d07fd/users/3577dce245ab44cb9c8c6ec6d3cb73bd/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:44.454749 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309a4e7c-9ac8-403c-9acb-fdfc016fd3bb None admin] RBAC: Authorizing `identity:create_grant(project_id=953c751045ff4e78a8125850cf1d07fd, user_id=3577dce245ab44cb9c8c6ec6d3cb73bd, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:44.455612 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa1e0e4-8ee3-4b1d-b3ed-a18c07fdef03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:44.456569 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa1e0e4-8ee3-4b1d-b3ed-a18c07fdef03 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:44.456746 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309a4e7c-9ac8-403c-9acb-fdfc016fd3bb None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:44.457222 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa1e0e4-8ee3-4b1d-b3ed-a18c07fdef03 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:44.457702 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa1e0e4-8ee3-4b1d-b3ed-a18c07fdef03 None admin] PATH_INFO: `/v3/users/0cd9488c4d2344c68b423e381c5ed58c` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:44.459362 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa1e0e4-8ee3-4b1d-b3ed-a18c07fdef03 None admin] RBAC: Authorizing `identity:get_user(user_id=0cd9488c4d2344c68b423e381c5ed58c)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:44.462253 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa1e0e4-8ee3-4b1d-b3ed-a18c07fdef03 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:44.464228 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 89/179] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Nov 2 03:20:44 2022] GET /identity/v3/users/0cd9488c4d2344c68b423e381c5ed58c => generated 283 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:44.469752 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa1e0e4-8ee3-4b1d-b3ed-a18c07fdef03 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:44.478928 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 91/180] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Nov 2 03:20:44 2022] PUT /identity/v3/projects/953c751045ff4e78a8125850cf1d07fd/users/3577dce245ab44cb9c8c6ec6d3cb73bd/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:20:44.488678 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309a4e7c-9ac8-403c-9acb-fdfc016fd3bb None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:44.575329 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:44.617825 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:44.695235 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309a4e7c-9ac8-403c-9acb-fdfc016fd3bb None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:44.701586 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa1e0e4-8ee3-4b1d-b3ed-a18c07fdef03 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:44.749007 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f7f79e-2eed-4f18-a53b-65d51268e646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:44.749555 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f7f79e-2eed-4f18-a53b-65d51268e646 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:44.749835 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f7f79e-2eed-4f18-a53b-65d51268e646 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:44.750105 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f7f79e-2eed-4f18-a53b-65d51268e646 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:44.764674 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d362039a-0fb0-49c6-b119-96a7c3fa4b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:44.765341 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362039a-0fb0-49c6-b119-96a7c3fa4b1a None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:44.767490 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362039a-0fb0-49c6-b119-96a7c3fa4b1a None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:44.769916 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d362039a-0fb0-49c6-b119-96a7c3fa4b1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:44.886531 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d362039a-0fb0-49c6-b119-96a7c3fa4b1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:44.888075 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d362039a-0fb0-49c6-b119-96a7c3fa4b1a None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:44.890017 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f7f79e-2eed-4f18-a53b-65d51268e646 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:44.891489 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f7f79e-2eed-4f18-a53b-65d51268e646 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:44.965534 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:44.981685 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:45.050697 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 92/181] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Nov 2 03:20:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 571 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 03:20:45.058154 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f7f79e-2eed-4f18-a53b-65d51268e646 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:45.079159 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 90/182] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Nov 2 03:20:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 610 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 03:20:45.084389 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d362039a-0fb0-49c6-b119-96a7c3fa4b1a None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:45.200187 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d362039a-0fb0-49c6-b119-96a7c3fa4b1a None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:45.235914 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f7f79e-2eed-4f18-a53b-65d51268e646 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:45.247460 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160cd391-9c66-41b5-a4cd-96b21ba0a6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:45.248347 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160cd391-9c66-41b5-a4cd-96b21ba0a6aa None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:45.248768 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160cd391-9c66-41b5-a4cd-96b21ba0a6aa None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:45.249181 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160cd391-9c66-41b5-a4cd-96b21ba0a6aa None admin] PATH_INFO: `/v3/projects/953c751045ff4e78a8125850cf1d07fd` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:45.251809 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160cd391-9c66-41b5-a4cd-96b21ba0a6aa None admin] RBAC: Authorizing `identity:get_project(project_id=953c751045ff4e78a8125850cf1d07fd)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:45.253408 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160cd391-9c66-41b5-a4cd-96b21ba0a6aa None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:45.255972 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 91/183] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Nov 2 03:20:45 2022] GET /identity/v3/projects/953c751045ff4e78a8125850cf1d07fd => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:20:45.262175 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160cd391-9c66-41b5-a4cd-96b21ba0a6aa None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:45.293386 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:45.316853 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecf155d-3ecf-4cf8-8983-f32de9d68dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:45.317724 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf155d-3ecf-4cf8-8983-f32de9d68dce None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:45.319894 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf155d-3ecf-4cf8-8983-f32de9d68dce None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:45.320372 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aecf155d-3ecf-4cf8-8983-f32de9d68dce None admin] PATH_INFO: `/v3/projects/953c751045ff4e78a8125850cf1d07fd` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:45.322842 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecf155d-3ecf-4cf8-8983-f32de9d68dce None admin] RBAC: Authorizing `identity:get_project(project_id=953c751045ff4e78a8125850cf1d07fd)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:45.324704 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aecf155d-3ecf-4cf8-8983-f32de9d68dce None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:45.328750 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 93/184] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Nov 2 03:20:45 2022] GET /identity/v3/projects/953c751045ff4e78a8125850cf1d07fd => generated 302 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:45.337000 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecf155d-3ecf-4cf8-8983-f32de9d68dce None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:45.351571 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:45.443943 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160cd391-9c66-41b5-a4cd-96b21ba0a6aa None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:45.487735 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aecf155d-3ecf-4cf8-8983-f32de9d68dce None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:45.506447 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb88bbac-54d8-4ca1-887e-786124041193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:45.507070 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb88bbac-54d8-4ca1-887e-786124041193 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:45.507351 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb88bbac-54d8-4ca1-887e-786124041193 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:45.507628 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb88bbac-54d8-4ca1-887e-786124041193 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:45.508438 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb88bbac-54d8-4ca1-887e-786124041193 None admin] RBAC: Adding query filter params (role.id=addc0db6b4ac4362b07fe21e14f89180, scope.project.id=953c751045ff4e78a8125850cf1d07fd, user.id=0cd9488c4d2344c68b423e381c5ed58c) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:20:45.508741 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb88bbac-54d8-4ca1-887e-786124041193 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:45.510189 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb88bbac-54d8-4ca1-887e-786124041193 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:45.529839 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c22eab-6a64-4fe7-adf4-bc7953543cc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:45.530560 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 92/185] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Nov 2 03:20:45 2022] GET /identity/v3/role_assignments?user.id=0cd9488c4d2344c68b423e381c5ed58c&scope.project.id=953c751045ff4e78a8125850cf1d07fd&role.id=addc0db6b4ac4362b07fe21e14f89180 => generated 608 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:45.530667 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c22eab-6a64-4fe7-adf4-bc7953543cc3 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:45.530740 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c22eab-6a64-4fe7-adf4-bc7953543cc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:45.530975 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c22eab-6a64-4fe7-adf4-bc7953543cc3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:45.531704 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c22eab-6a64-4fe7-adf4-bc7953543cc3 None admin] RBAC: Adding query filter params (role.id=f25d67f31fee4a92b3060e45a3fbe379, scope.project.id=953c751045ff4e78a8125850cf1d07fd, user.id=0cd9488c4d2344c68b423e381c5ed58c) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:20:45.532094 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c22eab-6a64-4fe7-adf4-bc7953543cc3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:45.533541 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c22eab-6a64-4fe7-adf4-bc7953543cc3 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:45.543905 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 94/186] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Nov 2 03:20:45 2022] GET /identity/v3/role_assignments?user.id=0cd9488c4d2344c68b423e381c5ed58c&scope.project.id=953c751045ff4e78a8125850cf1d07fd&role.id=f25d67f31fee4a92b3060e45a3fbe379 => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:46.215846 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498123bd-e67f-434c-b5d5-3188ed3c0864 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:46.216427 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498123bd-e67f-434c-b5d5-3188ed3c0864 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:46.216427 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498123bd-e67f-434c-b5d5-3188ed3c0864 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:46.217800 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 93/187] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:20:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:20:46.223763 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb8327d-3b72-4997-a402-8a06fa1fc1f2 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:46.224342 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb8327d-3b72-4997-a402-8a06fa1fc1f2 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:46.224342 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb8327d-3b72-4997-a402-8a06fa1fc1f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:46.244401 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-feb8327d-3b72-4997-a402-8a06fa1fc1f2 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:20:46.245621 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-feb8327d-3b72-4997-a402-8a06fa1fc1f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:20:46.314414 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 95/188] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:20:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 03:20:46.342181 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf1017-79b2-4234-9579-df8f14c70fcd None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:46.342466 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf1017-79b2-4234-9579-df8f14c70fcd None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:46.342730 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cf1017-79b2-4234-9579-df8f14c70fcd None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:46.344248 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 94/189] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:20:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:20:46.349161 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c8efcf-8d62-404e-a896-210104b704e9 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:46.349450 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c8efcf-8d62-404e-a896-210104b704e9 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:46.349728 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c8efcf-8d62-404e-a896-210104b704e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:46.357418 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:46.369973 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-88c8efcf-8d62-404e-a896-210104b704e9 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:20:46.371156 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-88c8efcf-8d62-404e-a896-210104b704e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:20:46.436270 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 96/190] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:20:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 03:20:46.579416 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cf1017-79b2-4234-9579-df8f14c70fcd None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:46.671575 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:46.683696 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cf1017-79b2-4234-9579-df8f14c70fcd None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:46.705016 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c8efcf-8d62-404e-a896-210104b704e9 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:46.719519 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f8d422-2f2d-4601-97a8-37c11ac109d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:46.720196 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f8d422-2f2d-4601-97a8-37c11ac109d6 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:46.720499 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f8d422-2f2d-4601-97a8-37c11ac109d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:46.720736 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f8d422-2f2d-4601-97a8-37c11ac109d6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:46.731650 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f8d422-2f2d-4601-97a8-37c11ac109d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:20:46.736515 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-50f8d422-2f2d-4601-97a8-37c11ac109d6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 03:20:46.736515 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:20:46.736515 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:20:46.736515 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:20:46.736515 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:20:46.736515 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:20:46.736515 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:20:46.736515 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:20:46.736515 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:20:46.736515 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:20:46.736515 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:20:46.736515 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:20:46.736515 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:20:46.736515 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:20:46.736515 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:20:46.736515 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:20:46.737636 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:20:46.737636 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:20:46.737636 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:20:46.737636 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:20:46.737636 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:20:46.737636 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:20:46.737636 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:20:46.737636 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:20:46.737636 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:20:46.737636 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:20:46.737636 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:20:46.737636 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:20:46.737636 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:20:46.737636 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:20:46.737636 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:20:46.737636 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:20:46.738433 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:20:46.738433 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:20:46.738433 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:20:46.738433 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:20:46.738433 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:20:46.738433 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:20:46.738433 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:20:46.738433 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:20:46.738433 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:20:46.738433 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:20:46.738433 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 03:20:46.738433 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:20:46.738433 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 95/191] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Nov 2 03:20:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:20:46.744795 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f8d422-2f2d-4601-97a8-37c11ac109d6 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:46.814005 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c8efcf-8d62-404e-a896-210104b704e9 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:46.849755 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f8d422-2f2d-4601-97a8-37c11ac109d6 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:46.850058 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c55ae1-3adb-4e24-b7af-0e7421220dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:46.850664 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c55ae1-3adb-4e24-b7af-0e7421220dc8 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:46.850958 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c55ae1-3adb-4e24-b7af-0e7421220dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:46.851273 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c55ae1-3adb-4e24-b7af-0e7421220dc8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:46.852979 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c55ae1-3adb-4e24-b7af-0e7421220dc8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:46.854422 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c55ae1-3adb-4e24-b7af-0e7421220dc8 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:46.856015 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 97/192] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:20:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:46.861295 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c55ae1-3adb-4e24-b7af-0e7421220dc8 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:46.864244 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:46.886394 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c92f882-33b4-42a1-8b51-24d1682b91a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:46.887016 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c92f882-33b4-42a1-8b51-24d1682b91a7 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:46.887277 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c92f882-33b4-42a1-8b51-24d1682b91a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:46.887546 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c92f882-33b4-42a1-8b51-24d1682b91a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:46.888245 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c92f882-33b4-42a1-8b51-24d1682b91a7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:20:46.888542 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c92f882-33b4-42a1-8b51-24d1682b91a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:46.889927 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c92f882-33b4-42a1-8b51-24d1682b91a7 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:46.898922 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 96/193] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Nov 2 03:20:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:20:46.901301 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f4307-2eeb-45a6-936c-39382553df08 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:46.901594 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f4307-2eeb-45a6-936c-39382553df08 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:46.901928 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92f4307-2eeb-45a6-936c-39382553df08 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:46.903320 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 97/194] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:20:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:20:46.904187 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92f4307-2eeb-45a6-936c-39382553df08 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:46.919944 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:46.965657 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c55ae1-3adb-4e24-b7af-0e7421220dc8 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:47.000449 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c04e900-bad0-4fd3-b1bf-79e1bbeccef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:47.001050 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c04e900-bad0-4fd3-b1bf-79e1bbeccef9 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:47.001325 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c04e900-bad0-4fd3-b1bf-79e1bbeccef9 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:47.001594 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c04e900-bad0-4fd3-b1bf-79e1bbeccef9 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:47.002250 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c04e900-bad0-4fd3-b1bf-79e1bbeccef9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:47.003671 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c04e900-bad0-4fd3-b1bf-79e1bbeccef9 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:47.006346 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92f4307-2eeb-45a6-936c-39382553df08 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:47.024351 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 98/195] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Nov 2 03:20:46 2022] POST /identity/v3/users => generated 293 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 03:20:47.029328 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd33846-d450-4d95-ac23-ccdf082cf5cc None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:47.029610 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd33846-d450-4d95-ac23-ccdf082cf5cc None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:47.029885 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd33846-d450-4d95-ac23-ccdf082cf5cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:47.046436 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc76f46c-fe1d-46b4-8fdb-b47a785b3cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:47.047284 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc76f46c-fe1d-46b4-8fdb-b47a785b3cd5 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:47.047611 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc76f46c-fe1d-46b4-8fdb-b47a785b3cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:47.047891 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc76f46c-fe1d-46b4-8fdb-b47a785b3cd5 None admin] PATH_INFO: `/v3/users/a245b7846fde482dbe5bde13da12b0e0` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:47.052060 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-dfd33846-d450-4d95-ac23-ccdf082cf5cc None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:20:47.053349 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-dfd33846-d450-4d95-ac23-ccdf082cf5cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:20:47.066273 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc76f46c-fe1d-46b4-8fdb-b47a785b3cd5 None admin] RBAC: Authorizing `identity:get_user(user_id=a245b7846fde482dbe5bde13da12b0e0)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:47.067667 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc76f46c-fe1d-46b4-8fdb-b47a785b3cd5 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:47.069724 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 98/196] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Nov 2 03:20:46 2022] GET /identity/v3/users/a245b7846fde482dbe5bde13da12b0e0 => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:47.075119 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc76f46c-fe1d-46b4-8fdb-b47a785b3cd5 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:47.120032 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:47.139245 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 99/197] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:20:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 03:20:47.154816 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:47.189567 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc76f46c-fe1d-46b4-8fdb-b47a785b3cd5 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:47.229181 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec52760f-463f-476a-a021-835e99ec1578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:47.229823 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec52760f-463f-476a-a021-835e99ec1578 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:47.230087 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec52760f-463f-476a-a021-835e99ec1578 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:47.230360 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec52760f-463f-476a-a021-835e99ec1578 None admin] PATH_INFO: `/v3/projects/953c751045ff4e78a8125850cf1d07fd` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:47.232141 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec52760f-463f-476a-a021-835e99ec1578 None admin] RBAC: Authorizing `identity:get_project(project_id=953c751045ff4e78a8125850cf1d07fd)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:47.233647 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec52760f-463f-476a-a021-835e99ec1578 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:47.235320 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 99/198] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Nov 2 03:20:47 2022] GET /identity/v3/projects/953c751045ff4e78a8125850cf1d07fd => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:47.241451 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd33846-d450-4d95-ac23-ccdf082cf5cc None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:47.348690 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd33846-d450-4d95-ac23-ccdf082cf5cc None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:47.386893 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4573af2-6903-4c66-a164-fef4b811e8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:47.387568 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4573af2-6903-4c66-a164-fef4b811e8a8 None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:47.387846 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4573af2-6903-4c66-a164-fef4b811e8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:47.388164 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4573af2-6903-4c66-a164-fef4b811e8a8 None admin] PATH_INFO: `/v3/projects/953c751045ff4e78a8125850cf1d07fd/users/a245b7846fde482dbe5bde13da12b0e0/roles/1966cb5a13b8421aad8823a14385cb43` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:47.390052 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4573af2-6903-4c66-a164-fef4b811e8a8 None admin] RBAC: Authorizing `identity:create_grant(project_id=953c751045ff4e78a8125850cf1d07fd, user_id=a245b7846fde482dbe5bde13da12b0e0, role_id=1966cb5a13b8421aad8823a14385cb43)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:47.391504 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4573af2-6903-4c66-a164-fef4b811e8a8 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:47.405963 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 100/199] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Nov 2 03:20:47 2022] PUT /identity/v3/projects/953c751045ff4e78a8125850cf1d07fd/users/a245b7846fde482dbe5bde13da12b0e0/roles/1966cb5a13b8421aad8823a14385cb43 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 03:20:47.409767 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec52760f-463f-476a-a021-835e99ec1578 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:47.529669 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec52760f-463f-476a-a021-835e99ec1578 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:47.566888 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f7df50-d80e-4a7d-bef4-055d3371c760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:47.567536 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f7df50-d80e-4a7d-bef4-055d3371c760 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:47.567811 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f7df50-d80e-4a7d-bef4-055d3371c760 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:47.568113 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f7df50-d80e-4a7d-bef4-055d3371c760 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:47.576625 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:47.581271 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f7df50-d80e-4a7d-bef4-055d3371c760 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:20:47.586236 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-f5f7df50-d80e-4a7d-bef4-055d3371c760 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 03:20:47.586236 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:20:47.586236 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:20:47.586236 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:20:47.586236 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:20:47.586236 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:20:47.586236 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:20:47.586236 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:20:47.586236 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:20:47.586236 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:20:47.586236 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:20:47.586236 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:20:47.586236 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:20:47.586236 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:20:47.586236 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:20:47.586236 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:20:47.587334 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:20:47.587334 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:20:47.587334 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:20:47.587334 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:20:47.587334 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:20:47.587334 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:20:47.587334 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:20:47.587334 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:20:47.587334 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:20:47.587334 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:20:47.587334 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:20:47.587334 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:20:47.587334 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:20:47.587334 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:20:47.587334 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:20:47.587334 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:20:47.588720 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:20:47.588720 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:20:47.588720 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:20:47.588720 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:20:47.588720 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:20:47.588720 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:20:47.588720 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:20:47.588720 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:20:47.588720 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:20:47.588720 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:20:47.588720 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 03:20:47.588720 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:20:47.588720 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 100/200] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Nov 2 03:20:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 03:20:47.593172 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4573af2-6903-4c66-a164-fef4b811e8a8 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:47.647583 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:47.703534 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4573af2-6903-4c66-a164-fef4b811e8a8 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:47.739370 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8576bcd-bb00-4992-9222-f74078053521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:47.740145 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8576bcd-bb00-4992-9222-f74078053521 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:47.740420 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8576bcd-bb00-4992-9222-f74078053521 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:47.740686 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8576bcd-bb00-4992-9222-f74078053521 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:47.741349 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8576bcd-bb00-4992-9222-f74078053521 None admin] RBAC: Adding query filter params (name=member) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:20:47.741644 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8576bcd-bb00-4992-9222-f74078053521 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:47.743540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8576bcd-bb00-4992-9222-f74078053521 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:47.752178 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 101/201] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Nov 2 03:20:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:47.757865 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f7df50-d80e-4a7d-bef4-055d3371c760 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:47.859205 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f7df50-d80e-4a7d-bef4-055d3371c760 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:47.893376 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4778ed-2823-4341-9d30-20d6b2679054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:47.894008 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4778ed-2823-4341-9d30-20d6b2679054 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:47.894288 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4778ed-2823-4341-9d30-20d6b2679054 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:47.894560 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4778ed-2823-4341-9d30-20d6b2679054 None admin] PATH_INFO: `/v3/users/3577dce245ab44cb9c8c6ec6d3cb73bd` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:47.895692 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4778ed-2823-4341-9d30-20d6b2679054 None admin] RBAC: Authorizing `identity:get_user(user_id=3577dce245ab44cb9c8c6ec6d3cb73bd)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:47.897176 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4778ed-2823-4341-9d30-20d6b2679054 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:47.899206 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 101/202] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Nov 2 03:20:47 2022] GET /identity/v3/users/3577dce245ab44cb9c8c6ec6d3cb73bd => generated 297 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:47.904649 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8576bcd-bb00-4992-9222-f74078053521 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:48.015103 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8576bcd-bb00-4992-9222-f74078053521 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:48.026456 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:48.051309 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff85f140-4aa9-4dab-b15c-8d813dc83952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:48.051928 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85f140-4aa9-4dab-b15c-8d813dc83952 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:48.052255 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85f140-4aa9-4dab-b15c-8d813dc83952 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:48.052532 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff85f140-4aa9-4dab-b15c-8d813dc83952 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:48.123634 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff85f140-4aa9-4dab-b15c-8d813dc83952 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:48.125279 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff85f140-4aa9-4dab-b15c-8d813dc83952 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:48.231084 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 102/203] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Nov 2 03:20:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 03:20:48.236459 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4778ed-2823-4341-9d30-20d6b2679054 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:48.301257 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:48.336627 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4778ed-2823-4341-9d30-20d6b2679054 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:48.375233 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d73fd1-2d24-498a-a0be-da927d2ef8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:48.375881 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d73fd1-2d24-498a-a0be-da927d2ef8da None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:48.376203 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d73fd1-2d24-498a-a0be-da927d2ef8da None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:48.376476 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d73fd1-2d24-498a-a0be-da927d2ef8da None admin] PATH_INFO: `/v3/projects/953c751045ff4e78a8125850cf1d07fd` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:48.377690 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d73fd1-2d24-498a-a0be-da927d2ef8da None admin] RBAC: Authorizing `identity:get_project(project_id=953c751045ff4e78a8125850cf1d07fd)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:48.379233 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d73fd1-2d24-498a-a0be-da927d2ef8da None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:48.381324 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 102/204] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Nov 2 03:20:48 2022] GET /identity/v3/projects/953c751045ff4e78a8125850cf1d07fd => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:48.386990 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff85f140-4aa9-4dab-b15c-8d813dc83952 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:48.399941 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:48.494181 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff85f140-4aa9-4dab-b15c-8d813dc83952 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:48.543972 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0508e43d-e3da-408d-a389-7c621e942d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:48.543972 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0508e43d-e3da-408d-a389-7c621e942d91 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:48.543972 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0508e43d-e3da-408d-a389-7c621e942d91 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:48.543972 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0508e43d-e3da-408d-a389-7c621e942d91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:48.544298 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0508e43d-e3da-408d-a389-7c621e942d91 None admin] RBAC: Adding query filter params (role.id=7a6df2ce66694822bce7983f7d988da5, scope.project.id=953c751045ff4e78a8125850cf1d07fd, user.id=3577dce245ab44cb9c8c6ec6d3cb73bd) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:20:48.544298 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0508e43d-e3da-408d-a389-7c621e942d91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:48.544440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0508e43d-e3da-408d-a389-7c621e942d91 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:48.550817 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 103/205] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Nov 2 03:20:48 2022] GET /identity/v3/role_assignments?user.id=3577dce245ab44cb9c8c6ec6d3cb73bd&scope.project.id=953c751045ff4e78a8125850cf1d07fd&role.id=7a6df2ce66694822bce7983f7d988da5 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:49.420785 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205cd288-f2dc-4b16-98c8-765390da0737 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:49.421309 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205cd288-f2dc-4b16-98c8-765390da0737 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:49.421309 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205cd288-f2dc-4b16-98c8-765390da0737 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:49.422681 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 103/206] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:20:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:20:49.427911 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c640d18d-c45e-4fba-bae4-9a0e06f928ab None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:49.428328 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c640d18d-c45e-4fba-bae4-9a0e06f928ab None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:49.428489 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c640d18d-c45e-4fba-bae4-9a0e06f928ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:49.447051 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-c640d18d-c45e-4fba-bae4-9a0e06f928ab None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:20:49.448252 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c640d18d-c45e-4fba-bae4-9a0e06f928ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:20:49.495640 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40882ed8-253c-4c02-a4f0-f7ce00010fc5 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:49.496003 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40882ed8-253c-4c02-a4f0-f7ce00010fc5 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:49.496237 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40882ed8-253c-4c02-a4f0-f7ce00010fc5 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:49.497586 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 104/207] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:20:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 03:20:49.502474 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de572e7d-0541-445a-86ce-ac26fd9d1815 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:49.502758 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de572e7d-0541-445a-86ce-ac26fd9d1815 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:49.503038 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de572e7d-0541-445a-86ce-ac26fd9d1815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:49.516734 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 104/208] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:20:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 03:20:49.526394 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-de572e7d-0541-445a-86ce-ac26fd9d1815 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:20:49.527823 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-de572e7d-0541-445a-86ce-ac26fd9d1815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:20:49.599080 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 105/209] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:20:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 03:20:49.675006 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb1c78a-ac89-47d8-a41a-b47780fd76bb None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:49.675283 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb1c78a-ac89-47d8-a41a-b47780fd76bb None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:49.675468 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb1c78a-ac89-47d8-a41a-b47780fd76bb None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:49.676993 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 105/210] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:20:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:20:49.682444 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f17c53c-b2ef-43db-8e23-d9dc4ecd3c51 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:49.682737 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f17c53c-b2ef-43db-8e23-d9dc4ecd3c51 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:49.683013 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f17c53c-b2ef-43db-8e23-d9dc4ecd3c51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:49.700901 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-8f17c53c-b2ef-43db-8e23-d9dc4ecd3c51 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:20:49.702065 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-8f17c53c-b2ef-43db-8e23-d9dc4ecd3c51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:20:49.761443 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:49.769804 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 106/211] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:20:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 03:20:49.784179 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb1c78a-ac89-47d8-a41a-b47780fd76bb None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:49.789614 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:49.865453 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f17c53c-b2ef-43db-8e23-d9dc4ecd3c51 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:49.889775 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb1c78a-ac89-47d8-a41a-b47780fd76bb None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:49.924952 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7ad465-9588-4a16-8a4a-00c1f4231296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:49.925952 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ad465-9588-4a16-8a4a-00c1f4231296 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:49.926076 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ad465-9588-4a16-8a4a-00c1f4231296 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:49.926314 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7ad465-9588-4a16-8a4a-00c1f4231296 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:49.936737 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7ad465-9588-4a16-8a4a-00c1f4231296 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:20:49.941629 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-0a7ad465-9588-4a16-8a4a-00c1f4231296 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 03:20:49.941629 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:20:49.941629 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:20:49.941629 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:20:49.941629 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:20:49.941629 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:20:49.941629 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:20:49.941629 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:20:49.941629 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:20:49.941629 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:20:49.941629 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:20:49.941629 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:20:49.941629 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:20:49.941629 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:20:49.941629 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:20:49.941629 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:20:49.944054 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:20:49.944054 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:20:49.944054 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:20:49.944054 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:20:49.944054 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:20:49.944054 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:20:49.944054 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:20:49.944054 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:20:49.944054 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:20:49.944054 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:20:49.944054 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:20:49.944054 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:20:49.944054 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:20:49.944054 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:20:49.944054 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:20:49.944054 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:20:49.945130 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:20:49.945130 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:20:49.945130 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:20:49.945130 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:20:49.945130 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:20:49.945130 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:20:49.945130 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:20:49.945130 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:20:49.945130 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:20:49.945130 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:20:49.945130 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 03:20:49.945130 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:20:49.945130 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 106/212] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Nov 2 03:20:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:20:49.948286 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7ad465-9588-4a16-8a4a-00c1f4231296 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:49.978738 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f17c53c-b2ef-43db-8e23-d9dc4ecd3c51 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:50.014709 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8e720f-e604-4f0a-a940-bd3d5b9271da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:50.015349 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e720f-e604-4f0a-a940-bd3d5b9271da None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:50.015624 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e720f-e604-4f0a-a940-bd3d5b9271da None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:50.015896 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8e720f-e604-4f0a-a940-bd3d5b9271da None admin] PATH_INFO: `/v3/domains/default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:50.017283 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8e720f-e604-4f0a-a940-bd3d5b9271da None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:50.018724 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8e720f-e604-4f0a-a940-bd3d5b9271da None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:50.019739 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:50.025393 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 107/213] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:20:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:20:50.026420 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8e720f-e604-4f0a-a940-bd3d5b9271da None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:50.062108 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7ad465-9588-4a16-8a4a-00c1f4231296 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:50.086868 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:50.094958 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4b0ea7-6e9a-4955-9c92-f8031050166b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:50.095569 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b0ea7-6e9a-4955-9c92-f8031050166b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:50.095843 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b0ea7-6e9a-4955-9c92-f8031050166b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:50.096144 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b0ea7-6e9a-4955-9c92-f8031050166b None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:50.096795 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4b0ea7-6e9a-4955-9c92-f8031050166b None admin] RBAC: Adding query filter params (name=member) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:20:50.097092 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4b0ea7-6e9a-4955-9c92-f8031050166b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:50.098463 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4b0ea7-6e9a-4955-9c92-f8031050166b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:50.107013 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 107/214] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Nov 2 03:20:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:20:50.107803 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4b0ea7-6e9a-4955-9c92-f8031050166b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:50.127860 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8e720f-e604-4f0a-a940-bd3d5b9271da None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:50.160720 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fc2038-fb02-4dde-ad82-b487c026706a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:50.161320 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fc2038-fb02-4dde-ad82-b487c026706a None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:50.161597 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fc2038-fb02-4dde-ad82-b487c026706a None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:50.161865 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fc2038-fb02-4dde-ad82-b487c026706a None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:50.162515 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fc2038-fb02-4dde-ad82-b487c026706a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:50.164481 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fc2038-fb02-4dde-ad82-b487c026706a None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:50.183986 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 108/215] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Nov 2 03:20:50 2022] POST /identity/v3/users => generated 293 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 03:20:50.193001 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fc2038-fb02-4dde-ad82-b487c026706a None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:50.213858 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4b0ea7-6e9a-4955-9c92-f8031050166b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:50.244290 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:50.254717 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636b29a5-e5f7-4d4e-91d5-0bd9af6d8acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:50.255371 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636b29a5-e5f7-4d4e-91d5-0bd9af6d8acb None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:50.255647 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636b29a5-e5f7-4d4e-91d5-0bd9af6d8acb None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:50.255918 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636b29a5-e5f7-4d4e-91d5-0bd9af6d8acb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:50.268109 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636b29a5-e5f7-4d4e-91d5-0bd9af6d8acb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:20:50.273636 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-636b29a5-e5f7-4d4e-91d5-0bd9af6d8acb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 03:20:50.273636 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:20:50.273636 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:20:50.273636 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:20:50.273636 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:20:50.273636 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:20:50.273636 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:20:50.273636 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:20:50.273636 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:20:50.273636 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:20:50.273636 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:20:50.273636 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:20:50.273636 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:20:50.273636 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:20:50.273636 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:20:50.273636 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:20:50.274592 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:20:50.274592 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:20:50.274592 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:20:50.274592 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:20:50.274592 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:20:50.274592 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:20:50.274592 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:20:50.274592 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:20:50.274592 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:20:50.274592 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:20:50.274592 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:20:50.274592 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:20:50.274592 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:20:50.274592 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:20:50.274592 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:20:50.274592 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:20:50.275497 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:20:50.275497 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:20:50.275497 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:20:50.275497 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:20:50.275497 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:20:50.275497 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:20:50.275497 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:20:50.275497 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:20:50.275497 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:20:50.275497 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:20:50.275497 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 03:20:50.275497 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:20:50.275497 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 108/216] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Nov 2 03:20:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:20:50.280912 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636b29a5-e5f7-4d4e-91d5-0bd9af6d8acb None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:50.324506 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fc2038-fb02-4dde-ad82-b487c026706a None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:50.342580 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:50.361775 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4661e4-0733-4391-a8a7-3aba3036b58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:50.362678 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4661e4-0733-4391-a8a7-3aba3036b58f None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:50.362950 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4661e4-0733-4391-a8a7-3aba3036b58f None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:50.363219 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4661e4-0733-4391-a8a7-3aba3036b58f None admin] PATH_INFO: `/v3/users/31063d0482e04f568659cd5a21a0a104` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:50.379372 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4661e4-0733-4391-a8a7-3aba3036b58f None admin] RBAC: Authorizing `identity:get_user(user_id=31063d0482e04f568659cd5a21a0a104)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:50.380866 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4661e4-0733-4391-a8a7-3aba3036b58f None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:50.382644 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 109/217] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Nov 2 03:20:50 2022] GET /identity/v3/users/31063d0482e04f568659cd5a21a0a104 => generated 293 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:50.388888 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4661e4-0733-4391-a8a7-3aba3036b58f None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:50.404969 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636b29a5-e5f7-4d4e-91d5-0bd9af6d8acb None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:50.443765 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fe97a3-0dcb-4d35-bfe3-653f56c6f64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:50.444397 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe97a3-0dcb-4d35-bfe3-653f56c6f64f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:50.444615 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe97a3-0dcb-4d35-bfe3-653f56c6f64f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:50.444874 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe97a3-0dcb-4d35-bfe3-653f56c6f64f None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:50.445580 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fe97a3-0dcb-4d35-bfe3-653f56c6f64f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:20:50.445883 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fe97a3-0dcb-4d35-bfe3-653f56c6f64f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:50.447264 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fe97a3-0dcb-4d35-bfe3-653f56c6f64f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:50.457393 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 109/218] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Nov 2 03:20:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:20:50.463480 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fe97a3-0dcb-4d35-bfe3-653f56c6f64f None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:50.509564 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4661e4-0733-4391-a8a7-3aba3036b58f None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:50.525293 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:50.557081 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922ab7c3-7574-400a-8d2f-4163a743bac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:50.557699 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ab7c3-7574-400a-8d2f-4163a743bac2 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:50.557975 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ab7c3-7574-400a-8d2f-4163a743bac2 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:50.558242 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922ab7c3-7574-400a-8d2f-4163a743bac2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:50.558966 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922ab7c3-7574-400a-8d2f-4163a743bac2 None admin] RBAC: Adding query filter params (role.id=7a6df2ce66694822bce7983f7d988da5, user.id=31063d0482e04f568659cd5a21a0a104, scope.system=all) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:20:50.559272 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922ab7c3-7574-400a-8d2f-4163a743bac2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:50.560754 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922ab7c3-7574-400a-8d2f-4163a743bac2 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:50.566883 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 110/219] 10.222.0.26 () {56 vars in 1308 bytes} [Wed Nov 2 03:20:50 2022] GET /identity/v3/role_assignments?user.id=31063d0482e04f568659cd5a21a0a104&scope.system=all&role.id=7a6df2ce66694822bce7983f7d988da5 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:20:50.592263 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fe97a3-0dcb-4d35-bfe3-653f56c6f64f None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:50.631709 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37ffb20-8150-4480-ba8d-afa3aedeb811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:50.632400 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ffb20-8150-4480-ba8d-afa3aedeb811 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:50.633240 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ffb20-8150-4480-ba8d-afa3aedeb811 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:50.633513 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37ffb20-8150-4480-ba8d-afa3aedeb811 None admin] PATH_INFO: `/v3/users/a245b7846fde482dbe5bde13da12b0e0` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:50.634756 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37ffb20-8150-4480-ba8d-afa3aedeb811 None admin] RBAC: Authorizing `identity:get_user(user_id=a245b7846fde482dbe5bde13da12b0e0)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:50.636259 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37ffb20-8150-4480-ba8d-afa3aedeb811 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:50.638254 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 110/220] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Nov 2 03:20:50 2022] GET /identity/v3/users/a245b7846fde482dbe5bde13da12b0e0 => generated 297 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:20:50.643528 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922ab7c3-7574-400a-8d2f-4163a743bac2 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:50.691476 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:50.758265 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922ab7c3-7574-400a-8d2f-4163a743bac2 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:50.793333 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d10874-2dd9-4fd9-b340-ab4e2edb4a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:50.793945 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d10874-2dd9-4fd9-b340-ab4e2edb4a81 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:50.794223 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d10874-2dd9-4fd9-b340-ab4e2edb4a81 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:50.794493 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d10874-2dd9-4fd9-b340-ab4e2edb4a81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:50.864875 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d10874-2dd9-4fd9-b340-ab4e2edb4a81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:50.866339 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d10874-2dd9-4fd9-b340-ab4e2edb4a81 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:50.907617 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:50.969849 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 111/221] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Nov 2 03:20:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 03:20:50.975656 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37ffb20-8150-4480-ba8d-afa3aedeb811 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:51.081768 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37ffb20-8150-4480-ba8d-afa3aedeb811 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:51.099275 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:51.120389 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967d28d2-5e4b-4a7d-b835-739585473cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:51.121387 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967d28d2-5e4b-4a7d-b835-739585473cc0 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:51.121860 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967d28d2-5e4b-4a7d-b835-739585473cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:51.122337 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967d28d2-5e4b-4a7d-b835-739585473cc0 None admin] PATH_INFO: `/v3/projects/953c751045ff4e78a8125850cf1d07fd` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:51.123653 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967d28d2-5e4b-4a7d-b835-739585473cc0 None admin] RBAC: Authorizing `identity:get_project(project_id=953c751045ff4e78a8125850cf1d07fd)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:51.125328 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967d28d2-5e4b-4a7d-b835-739585473cc0 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:51.127620 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 111/222] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Nov 2 03:20:50 2022] GET /identity/v3/projects/953c751045ff4e78a8125850cf1d07fd => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:51.133351 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d10874-2dd9-4fd9-b340-ab4e2edb4a81 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:51.236088 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d10874-2dd9-4fd9-b340-ab4e2edb4a81 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:51.271641 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604920af-9951-468e-b8b2-f30740a71810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:51.272373 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604920af-9951-468e-b8b2-f30740a71810 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:51.272654 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604920af-9951-468e-b8b2-f30740a71810 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:51.272926 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604920af-9951-468e-b8b2-f30740a71810 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:51.274225 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604920af-9951-468e-b8b2-f30740a71810 None admin] RBAC: Adding query filter params (role.id=1966cb5a13b8421aad8823a14385cb43, scope.project.id=953c751045ff4e78a8125850cf1d07fd, user.id=a245b7846fde482dbe5bde13da12b0e0) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:20:51.274534 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604920af-9951-468e-b8b2-f30740a71810 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:51.276057 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604920af-9951-468e-b8b2-f30740a71810 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:51.283134 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 112/223] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Nov 2 03:20:51 2022] GET /identity/v3/role_assignments?user.id=a245b7846fde482dbe5bde13da12b0e0&scope.project.id=953c751045ff4e78a8125850cf1d07fd&role.id=1966cb5a13b8421aad8823a14385cb43 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:52.546640 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dea090-17b4-418e-a7c5-29ff65b329fe None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:52.547119 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dea090-17b4-418e-a7c5-29ff65b329fe None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:52.547194 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dea090-17b4-418e-a7c5-29ff65b329fe None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:52.548544 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 112/224] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:20:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:20:52.553793 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e4d820-afd0-402d-8e54-635f090efe4f None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:52.554174 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e4d820-afd0-402d-8e54-635f090efe4f None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:52.554336 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e4d820-afd0-402d-8e54-635f090efe4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:52.563213 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:52.574761 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-29e4d820-afd0-402d-8e54-635f090efe4f None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:20:52.575927 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-29e4d820-afd0-402d-8e54-635f090efe4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:20:52.577835 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf300d-afc4-4a39-989f-f2ba5ebb1aad None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:52.578118 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf300d-afc4-4a39-989f-f2ba5ebb1aad None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:52.578388 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2faf300d-afc4-4a39-989f-f2ba5ebb1aad None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:52.579860 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 113/225] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:20:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 03:20:52.584900 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e540a-dd1b-4c23-8cc3-662ca7fec1e8 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:52.585358 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e540a-dd1b-4c23-8cc3-662ca7fec1e8 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:52.585802 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e540a-dd1b-4c23-8cc3-662ca7fec1e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:52.604476 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-e66e540a-dd1b-4c23-8cc3-662ca7fec1e8 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:20:52.605654 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e66e540a-dd1b-4c23-8cc3-662ca7fec1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:20:52.645544 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 113/226] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:20:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 03:20:52.674210 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 114/227] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:20:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 03:20:52.789718 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3279630-342e-44cc-89fe-0fe9024ef559 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:52.790006 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3279630-342e-44cc-89fe-0fe9024ef559 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:52.790204 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3279630-342e-44cc-89fe-0fe9024ef559 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:52.791565 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 114/228] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:20:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:20:52.796653 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307078ff-d5f3-4a8c-a3cb-43c998b124de None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:52.797055 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307078ff-d5f3-4a8c-a3cb-43c998b124de None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:52.797401 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307078ff-d5f3-4a8c-a3cb-43c998b124de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:52.816465 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-307078ff-d5f3-4a8c-a3cb-43c998b124de None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:20:52.817749 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-307078ff-d5f3-4a8c-a3cb-43c998b124de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:20:52.885367 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 115/229] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:20:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 03:20:52.909969 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3279630-342e-44cc-89fe-0fe9024ef559 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:52.943379 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307078ff-d5f3-4a8c-a3cb-43c998b124de None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:52.995980 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:53.018814 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3279630-342e-44cc-89fe-0fe9024ef559 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:53.053559 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307078ff-d5f3-4a8c-a3cb-43c998b124de None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:53.053836 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e51b1f1-fbec-4671-90bd-7c841e92eada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:53.054405 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e51b1f1-fbec-4671-90bd-7c841e92eada None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:53.054728 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e51b1f1-fbec-4671-90bd-7c841e92eada None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:53.055008 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e51b1f1-fbec-4671-90bd-7c841e92eada None admin] PATH_INFO: `/v3/domains/default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:53.056146 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e51b1f1-fbec-4671-90bd-7c841e92eada None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:53.057576 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e51b1f1-fbec-4671-90bd-7c841e92eada None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:53.059294 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 115/230] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:20:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:53.064438 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e51b1f1-fbec-4671-90bd-7c841e92eada None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:53.066976 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:53.091311 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3598ea65-0164-4f12-a51f-108b238e5484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:53.091943 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598ea65-0164-4f12-a51f-108b238e5484 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:53.092353 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598ea65-0164-4f12-a51f-108b238e5484 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:53.092619 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3598ea65-0164-4f12-a51f-108b238e5484 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:53.103732 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3598ea65-0164-4f12-a51f-108b238e5484 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:20:53.108934 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-3598ea65-0164-4f12-a51f-108b238e5484 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 03:20:53.108934 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:20:53.108934 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:20:53.108934 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:20:53.108934 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:20:53.108934 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:20:53.108934 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:20:53.108934 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:20:53.108934 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:20:53.108934 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:20:53.108934 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:20:53.108934 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:20:53.108934 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:20:53.108934 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:20:53.108934 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:20:53.108934 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:20:53.110031 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:20:53.110031 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:20:53.110031 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:20:53.110031 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:20:53.110031 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:20:53.110031 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:20:53.110031 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:20:53.110031 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:20:53.110031 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:20:53.110031 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:20:53.110031 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:20:53.110031 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:20:53.110031 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:20:53.110031 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:20:53.110031 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:20:53.110031 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:20:53.110828 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:20:53.110828 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:20:53.110828 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:20:53.110828 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:20:53.110828 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:20:53.110828 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:20:53.110828 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:20:53.110828 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:20:53.110828 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:20:53.110828 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:20:53.110828 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 03:20:53.110828 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:20:53.110828 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 116/231] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Nov 2 03:20:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 03:20:53.116060 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3598ea65-0164-4f12-a51f-108b238e5484 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:53.168619 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e51b1f1-fbec-4671-90bd-7c841e92eada None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:53.202093 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66804b1-87cf-4dd8-8ac5-95cbdd27001c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:53.202762 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66804b1-87cf-4dd8-8ac5-95cbdd27001c None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:53.203047 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66804b1-87cf-4dd8-8ac5-95cbdd27001c None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:53.203315 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66804b1-87cf-4dd8-8ac5-95cbdd27001c None admin] PATH_INFO: `/v3/groups` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:53.203941 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66804b1-87cf-4dd8-8ac5-95cbdd27001c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:53.205487 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66804b1-87cf-4dd8-8ac5-95cbdd27001c None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:53.219187 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3598ea65-0164-4f12-a51f-108b238e5484 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:53.220847 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 116/232] 10.222.0.26 () {60 vars in 1134 bytes} [Wed Nov 2 03:20:53 2022] POST /identity/v3/groups => generated 229 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 03:20:53.221502 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:53.224963 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66804b1-87cf-4dd8-8ac5-95cbdd27001c None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:53.260725 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7f84bc-5cd2-4bd2-aa29-88adf007a853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:53.261763 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f84bc-5cd2-4bd2-aa29-88adf007a853 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:53.262267 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f84bc-5cd2-4bd2-aa29-88adf007a853 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:53.262730 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7f84bc-5cd2-4bd2-aa29-88adf007a853 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:53.263602 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7f84bc-5cd2-4bd2-aa29-88adf007a853 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:20:53.264148 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7f84bc-5cd2-4bd2-aa29-88adf007a853 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:53.266754 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb7f84bc-5cd2-4bd2-aa29-88adf007a853 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:53.278441 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 117/233] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Nov 2 03:20:53 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:53.284836 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7f84bc-5cd2-4bd2-aa29-88adf007a853 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:53.297511 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:53.346101 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66804b1-87cf-4dd8-8ac5-95cbdd27001c None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:53.384562 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ea67ff-94dd-4873-a1cb-96f787f69848 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:53.385482 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea67ff-94dd-4873-a1cb-96f787f69848 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:53.385763 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea67ff-94dd-4873-a1cb-96f787f69848 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:53.386037 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ea67ff-94dd-4873-a1cb-96f787f69848 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:53.399229 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7f84bc-5cd2-4bd2-aa29-88adf007a853 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:53.401643 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ea67ff-94dd-4873-a1cb-96f787f69848 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:20:53.407786 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-11ea67ff-94dd-4873-a1cb-96f787f69848 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 03:20:53.407786 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:20:53.407786 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:20:53.407786 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:20:53.407786 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:20:53.407786 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:20:53.407786 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:20:53.407786 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:20:53.407786 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:20:53.407786 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:20:53.407786 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:20:53.407786 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:20:53.407786 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:20:53.407786 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:20:53.407786 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:20:53.407786 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:20:53.408698 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:20:53.408698 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:20:53.408698 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:20:53.408698 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:20:53.408698 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:20:53.408698 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:20:53.408698 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:20:53.408698 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:20:53.408698 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:20:53.408698 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:20:53.408698 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:20:53.408698 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:20:53.408698 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:20:53.408698 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:20:53.408698 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:20:53.408698 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:20:53.409486 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:20:53.409486 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:20:53.409486 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:20:53.409486 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:20:53.409486 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:20:53.409486 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:20:53.409486 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:20:53.409486 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:20:53.409486 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:20:53.409486 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:20:53.409486 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 03:20:53.409486 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:20:53.409486 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 117/234] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Nov 2 03:20:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:20:53.414405 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ea67ff-94dd-4873-a1cb-96f787f69848 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:53.436801 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9186ba9-8db5-4d3a-abcc-c468d5e92da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:53.437447 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9186ba9-8db5-4d3a-abcc-c468d5e92da1 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:53.437723 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9186ba9-8db5-4d3a-abcc-c468d5e92da1 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:53.437994 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9186ba9-8db5-4d3a-abcc-c468d5e92da1 None admin] PATH_INFO: `/v3/users/d79625965f1e42bb94193a10c3c06e2a` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:53.452493 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9186ba9-8db5-4d3a-abcc-c468d5e92da1 None admin] RBAC: Authorizing `identity:get_user(user_id=d79625965f1e42bb94193a10c3c06e2a)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:53.453973 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9186ba9-8db5-4d3a-abcc-c468d5e92da1 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:53.455629 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 118/235] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Nov 2 03:20:53 2022] GET /identity/v3/users/d79625965f1e42bb94193a10c3c06e2a => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:20:53.461897 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9186ba9-8db5-4d3a-abcc-c468d5e92da1 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:53.490370 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:53.531837 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ea67ff-94dd-4873-a1cb-96f787f69848 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:53.557885 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:53.570883 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcafb50-98f5-4bd5-8c09-8a11944f04bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:53.571499 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcafb50-98f5-4bd5-8c09-8a11944f04bd None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:53.571776 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcafb50-98f5-4bd5-8c09-8a11944f04bd None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:53.572084 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bcafb50-98f5-4bd5-8c09-8a11944f04bd None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:53.572747 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcafb50-98f5-4bd5-8c09-8a11944f04bd None admin] RBAC: Adding query filter params (name=member) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:20:53.573054 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcafb50-98f5-4bd5-8c09-8a11944f04bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:53.574470 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bcafb50-98f5-4bd5-8c09-8a11944f04bd None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:53.584385 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 118/236] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Nov 2 03:20:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:20:53.586708 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9186ba9-8db5-4d3a-abcc-c468d5e92da1 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:53.590812 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcafb50-98f5-4bd5-8c09-8a11944f04bd None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:53.627625 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb785d6c-9d15-4abb-b6c3-ed56cef0c406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:53.628197 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb785d6c-9d15-4abb-b6c3-ed56cef0c406 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:53.628481 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb785d6c-9d15-4abb-b6c3-ed56cef0c406 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:53.628759 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb785d6c-9d15-4abb-b6c3-ed56cef0c406 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:53.629489 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb785d6c-9d15-4abb-b6c3-ed56cef0c406 None admin] RBAC: Adding query filter params (role.id=1966cb5a13b8421aad8823a14385cb43, user.id=d79625965f1e42bb94193a10c3c06e2a, scope.system=all) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:20:53.629799 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb785d6c-9d15-4abb-b6c3-ed56cef0c406 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:53.632243 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb785d6c-9d15-4abb-b6c3-ed56cef0c406 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:53.637829 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 119/237] 10.222.0.26 () {56 vars in 1308 bytes} [Wed Nov 2 03:20:53 2022] GET /identity/v3/role_assignments?user.id=d79625965f1e42bb94193a10c3c06e2a&scope.system=all&role.id=1966cb5a13b8421aad8823a14385cb43 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:20:53.708106 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bcafb50-98f5-4bd5-8c09-8a11944f04bd None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:53.747483 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065f3abb-f79d-4c1f-bfb3-8dc25af227c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:53.748615 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065f3abb-f79d-4c1f-bfb3-8dc25af227c3 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:53.749099 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065f3abb-f79d-4c1f-bfb3-8dc25af227c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:53.749584 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065f3abb-f79d-4c1f-bfb3-8dc25af227c3 None admin] PATH_INFO: `/v3/users/31063d0482e04f568659cd5a21a0a104` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:53.751000 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065f3abb-f79d-4c1f-bfb3-8dc25af227c3 None admin] RBAC: Authorizing `identity:get_user(user_id=31063d0482e04f568659cd5a21a0a104)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:53.752735 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065f3abb-f79d-4c1f-bfb3-8dc25af227c3 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:53.755037 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 119/238] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Nov 2 03:20:53 2022] GET /identity/v3/users/31063d0482e04f568659cd5a21a0a104 => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:20:53.760557 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb785d6c-9d15-4abb-b6c3-ed56cef0c406 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:53.855715 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:53.869017 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb785d6c-9d15-4abb-b6c3-ed56cef0c406 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:53.903434 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e27fc24-ed45-433c-b515-6ae9ae20a91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:53.904040 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e27fc24-ed45-433c-b515-6ae9ae20a91e None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:53.904325 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e27fc24-ed45-433c-b515-6ae9ae20a91e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:53.904601 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e27fc24-ed45-433c-b515-6ae9ae20a91e None admin] PATH_INFO: `/v3/system/users/31063d0482e04f568659cd5a21a0a104/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:53.906017 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e27fc24-ed45-433c-b515-6ae9ae20a91e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=31063d0482e04f568659cd5a21a0a104, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:53.907393 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e27fc24-ed45-433c-b515-6ae9ae20a91e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:53.917563 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 120/239] 10.222.0.26 () {58 vars in 1334 bytes} [Wed Nov 2 03:20:53 2022] PUT /identity/v3/system/users/31063d0482e04f568659cd5a21a0a104/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:20:55.400801 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0466aa0b-d4b2-477d-9b4f-f379d7da7e92 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:55.401321 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0466aa0b-d4b2-477d-9b4f-f379d7da7e92 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:55.401428 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0466aa0b-d4b2-477d-9b4f-f379d7da7e92 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:55.402857 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 120/240] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:20:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:20:55.408155 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec998099-443f-4d61-8391-f1eacc5cc1d8 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:55.408542 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec998099-443f-4d61-8391-f1eacc5cc1d8 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:55.408738 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec998099-443f-4d61-8391-f1eacc5cc1d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:55.427385 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-ec998099-443f-4d61-8391-f1eacc5cc1d8 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:20:55.428606 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ec998099-443f-4d61-8391-f1eacc5cc1d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:20:55.496277 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 121/241] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:20:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 03:20:55.766542 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0466aa0b-d4b2-477d-9b4f-f379d7da7e92 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:55.788145 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236c95cb-0e99-4523-b9fd-d6b995f8634c None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:55.788445 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236c95cb-0e99-4523-b9fd-d6b995f8634c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:55.788712 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236c95cb-0e99-4523-b9fd-d6b995f8634c None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:55.790202 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 122/242] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:20:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:20:55.795349 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401f45eb-e29e-4863-b4f0-e4da818faee2 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:55.795634 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401f45eb-e29e-4863-b4f0-e4da818faee2 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:55.796254 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401f45eb-e29e-4863-b4f0-e4da818faee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:55.815170 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-401f45eb-e29e-4863-b4f0-e4da818faee2 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:20:55.816372 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-401f45eb-e29e-4863-b4f0-e4da818faee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:20:55.823747 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:55.878463 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0466aa0b-d4b2-477d-9b4f-f379d7da7e92 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:55.890232 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 123/243] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:20:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 03:20:55.918175 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c08080-a04e-4dd4-8840-dfc037633e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:55.919116 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c08080-a04e-4dd4-8840-dfc037633e94 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:55.919544 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c08080-a04e-4dd4-8840-dfc037633e94 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:55.920010 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c08080-a04e-4dd4-8840-dfc037633e94 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:55.921349 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c08080-a04e-4dd4-8840-dfc037633e94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:55.923057 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c08080-a04e-4dd4-8840-dfc037633e94 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:55.924856 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 121/244] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:20:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:55.930445 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401f45eb-e29e-4863-b4f0-e4da818faee2 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:55.947056 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:56.038378 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401f45eb-e29e-4863-b4f0-e4da818faee2 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:56.056186 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b372c6-8fc9-4a1f-83dd-e8e983e81e03 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:56.056493 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b372c6-8fc9-4a1f-83dd-e8e983e81e03 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:56.056742 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b372c6-8fc9-4a1f-83dd-e8e983e81e03 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:56.058232 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 122/245] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:20:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 03:20:56.063300 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7656e6d-0fc8-4ef4-bd41-2400dfdfb6f7 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:56.063585 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7656e6d-0fc8-4ef4-bd41-2400dfdfb6f7 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:56.063861 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7656e6d-0fc8-4ef4-bd41-2400dfdfb6f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:56.076958 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dc41d2-ac54-41b4-83bd-d37b32c9dcd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:56.077845 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc41d2-ac54-41b4-83bd-d37b32c9dcd9 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:56.078307 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc41d2-ac54-41b4-83bd-d37b32c9dcd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:56.078760 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4dc41d2-ac54-41b4-83bd-d37b32c9dcd9 None admin] PATH_INFO: `/v3/groups` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:56.079575 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dc41d2-ac54-41b4-83bd-d37b32c9dcd9 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:56.081231 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4dc41d2-ac54-41b4-83bd-d37b32c9dcd9 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:56.082645 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-a7656e6d-0fc8-4ef4-bd41-2400dfdfb6f7 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:20:56.083848 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a7656e6d-0fc8-4ef4-bd41-2400dfdfb6f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:20:56.099227 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 124/246] 10.222.0.26 () {60 vars in 1134 bytes} [Wed Nov 2 03:20:55 2022] POST /identity/v3/groups => generated 226 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 03:20:56.163317 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dc41d2-ac54-41b4-83bd-d37b32c9dcd9 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:56.171270 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 123/247] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:20:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 03:20:56.244107 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:56.279937 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4dc41d2-ac54-41b4-83bd-d37b32c9dcd9 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:56.315613 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a2ffc8-a421-4b9e-9d24-20f64f3c2e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:56.316229 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a2ffc8-a421-4b9e-9d24-20f64f3c2e31 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:56.316508 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a2ffc8-a421-4b9e-9d24-20f64f3c2e31 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:56.316772 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a2ffc8-a421-4b9e-9d24-20f64f3c2e31 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:56.328330 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a2ffc8-a421-4b9e-9d24-20f64f3c2e31 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:20:56.333631 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-66a2ffc8-a421-4b9e-9d24-20f64f3c2e31 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 03:20:56.333631 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:20:56.333631 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:20:56.333631 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:20:56.333631 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:20:56.333631 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:20:56.333631 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:20:56.333631 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:20:56.333631 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:20:56.333631 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:20:56.333631 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:20:56.333631 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:20:56.333631 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:20:56.333631 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:20:56.333631 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:20:56.333631 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:20:56.334557 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:20:56.334557 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:20:56.334557 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:20:56.334557 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:20:56.334557 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:20:56.334557 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:20:56.334557 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:20:56.334557 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:20:56.334557 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:20:56.334557 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:20:56.334557 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:20:56.334557 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:20:56.334557 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:20:56.334557 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:20:56.334557 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:20:56.334557 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:20:56.335373 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:20:56.335373 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:20:56.335373 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:20:56.335373 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:20:56.335373 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:20:56.335373 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:20:56.335373 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:20:56.335373 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:20:56.335373 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:20:56.335373 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:20:56.335373 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 03:20:56.335373 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:20:56.335373 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 125/248] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Nov 2 03:20:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 03:20:56.342033 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7656e6d-0fc8-4ef4-bd41-2400dfdfb6f7 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:56.401076 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:56.444674 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a2ffc8-a421-4b9e-9d24-20f64f3c2e31 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:56.461455 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7656e6d-0fc8-4ef4-bd41-2400dfdfb6f7 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:56.508691 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01384eb4-f347-4af4-b88b-d05810eba903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:56.509337 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01384eb4-f347-4af4-b88b-d05810eba903 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:56.509617 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01384eb4-f347-4af4-b88b-d05810eba903 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:56.509885 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01384eb4-f347-4af4-b88b-d05810eba903 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:56.510550 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01384eb4-f347-4af4-b88b-d05810eba903 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:20:56.510846 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01384eb4-f347-4af4-b88b-d05810eba903 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:56.512244 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01384eb4-f347-4af4-b88b-d05810eba903 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:56.521964 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 124/249] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Nov 2 03:20:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:56.528486 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01384eb4-f347-4af4-b88b-d05810eba903 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:56.588079 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a2ffc8-a421-4b9e-9d24-20f64f3c2e31 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:56.638589 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:56.649174 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce69af5a-73d0-4a00-b222-567712ccf57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:56.650129 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce69af5a-73d0-4a00-b222-567712ccf57b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:56.650633 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce69af5a-73d0-4a00-b222-567712ccf57b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:56.651643 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce69af5a-73d0-4a00-b222-567712ccf57b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:56.667736 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01384eb4-f347-4af4-b88b-d05810eba903 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:56.668889 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce69af5a-73d0-4a00-b222-567712ccf57b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:20:56.675436 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-ce69af5a-73d0-4a00-b222-567712ccf57b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 03:20:56.675436 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:20:56.675436 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:20:56.675436 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:20:56.675436 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:20:56.675436 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:20:56.675436 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:20:56.675436 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:20:56.675436 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:20:56.675436 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:20:56.675436 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:20:56.675436 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:20:56.675436 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:20:56.675436 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:20:56.675436 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:20:56.675436 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:20:56.676475 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:20:56.676475 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:20:56.676475 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:20:56.676475 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:20:56.676475 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:20:56.676475 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:20:56.676475 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:20:56.676475 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:20:56.676475 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:20:56.676475 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:20:56.676475 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:20:56.676475 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:20:56.676475 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:20:56.676475 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:20:56.676475 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:20:56.676475 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:20:56.677270 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:20:56.677270 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:20:56.677270 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:20:56.677270 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:20:56.677270 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:20:56.677270 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:20:56.677270 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:20:56.677270 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:20:56.677270 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:20:56.677270 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:20:56.677270 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 03:20:56.677270 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:20:56.680475 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 126/250] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Nov 2 03:20:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 03:20:56.688619 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:56.690169 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce69af5a-73d0-4a00-b222-567712ccf57b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:56.727854 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1988fd0-85b6-4394-8a4b-996c274d3dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:56.728734 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1988fd0-85b6-4394-8a4b-996c274d3dac None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:56.729427 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1988fd0-85b6-4394-8a4b-996c274d3dac None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:56.729705 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1988fd0-85b6-4394-8a4b-996c274d3dac None admin] PATH_INFO: `/v3/users/d79625965f1e42bb94193a10c3c06e2a` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:56.731081 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1988fd0-85b6-4394-8a4b-996c274d3dac None admin] RBAC: Authorizing `identity:get_user(user_id=d79625965f1e42bb94193a10c3c06e2a)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:56.732837 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1988fd0-85b6-4394-8a4b-996c274d3dac None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:56.736082 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 125/251] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Nov 2 03:20:56 2022] GET /identity/v3/users/d79625965f1e42bb94193a10c3c06e2a => generated 293 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:20:56.744384 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1988fd0-85b6-4394-8a4b-996c274d3dac None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:56.822085 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce69af5a-73d0-4a00-b222-567712ccf57b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:56.870788 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f673e98-03aa-4b1d-8b89-860934ece46e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:56.871348 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f673e98-03aa-4b1d-8b89-860934ece46e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:56.871708 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f673e98-03aa-4b1d-8b89-860934ece46e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:56.872011 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f673e98-03aa-4b1d-8b89-860934ece46e None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:56.872801 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f673e98-03aa-4b1d-8b89-860934ece46e None admin] RBAC: Adding query filter params (name=member) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:20:56.873094 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f673e98-03aa-4b1d-8b89-860934ece46e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:56.876175 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f673e98-03aa-4b1d-8b89-860934ece46e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:56.893592 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 127/252] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Nov 2 03:20:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:56.895114 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f673e98-03aa-4b1d-8b89-860934ece46e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:56.900005 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1988fd0-85b6-4394-8a4b-996c274d3dac None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:56.946983 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e2c814-f892-426f-8325-ce06d49b1df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:56.947651 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2c814-f892-426f-8325-ce06d49b1df4 None admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:56.948053 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2c814-f892-426f-8325-ce06d49b1df4 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:56.948339 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e2c814-f892-426f-8325-ce06d49b1df4 None admin] PATH_INFO: `/v3/system/users/d79625965f1e42bb94193a10c3c06e2a/roles/1966cb5a13b8421aad8823a14385cb43` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:56.950847 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e2c814-f892-426f-8325-ce06d49b1df4 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d79625965f1e42bb94193a10c3c06e2a, role_id=1966cb5a13b8421aad8823a14385cb43)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:56.952358 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e2c814-f892-426f-8325-ce06d49b1df4 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:56.963754 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 126/253] 10.222.0.26 () {58 vars in 1334 bytes} [Wed Nov 2 03:20:56 2022] PUT /identity/v3/system/users/d79625965f1e42bb94193a10c3c06e2a/roles/1966cb5a13b8421aad8823a14385cb43 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:20:56.975483 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:57.035717 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f673e98-03aa-4b1d-8b89-860934ece46e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:57.078271 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762b55e0-f9d0-4154-b1a6-1681cc504279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:57.078910 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b55e0-f9d0-4154-b1a6-1681cc504279 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:57.079188 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b55e0-f9d0-4154-b1a6-1681cc504279 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:57.079459 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b55e0-f9d0-4154-b1a6-1681cc504279 None admin] PATH_INFO: `/v3/users/31063d0482e04f568659cd5a21a0a104` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:57.080594 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762b55e0-f9d0-4154-b1a6-1681cc504279 None admin] RBAC: Authorizing `identity:get_user(user_id=31063d0482e04f568659cd5a21a0a104)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:57.082047 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762b55e0-f9d0-4154-b1a6-1681cc504279 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:57.084909 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 128/254] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Nov 2 03:20:56 2022] GET /identity/v3/users/31063d0482e04f568659cd5a21a0a104 => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:57.089999 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e2c814-f892-426f-8325-ce06d49b1df4 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:57.145407 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:57.217444 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e2c814-f892-426f-8325-ce06d49b1df4 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:57.268793 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b7fd17-be0a-4b08-934b-a964362d2312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:57.269430 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b7fd17-be0a-4b08-934b-a964362d2312 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:57.269710 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b7fd17-be0a-4b08-934b-a964362d2312 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:57.269977 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b7fd17-be0a-4b08-934b-a964362d2312 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:57.270722 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b7fd17-be0a-4b08-934b-a964362d2312 None admin] RBAC: Adding query filter params (role.id=7a6df2ce66694822bce7983f7d988da5, user.id=31063d0482e04f568659cd5a21a0a104, scope.system=all) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:20:57.271025 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b7fd17-be0a-4b08-934b-a964362d2312 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:57.272515 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b7fd17-be0a-4b08-934b-a964362d2312 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:57.286821 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 127/255] 10.222.0.26 () {56 vars in 1308 bytes} [Wed Nov 2 03:20:57 2022] GET /identity/v3/role_assignments?user.id=31063d0482e04f568659cd5a21a0a104&scope.system=all&role.id=7a6df2ce66694822bce7983f7d988da5 => generated 510 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:58.665976 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36a585-5155-4aa9-b943-9c81eec116ef None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:58.666503 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36a585-5155-4aa9-b943-9c81eec116ef None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:58.666503 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c36a585-5155-4aa9-b943-9c81eec116ef None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:58.668014 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 129/256] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:20:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:20:58.673485 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af05a2f-8955-4ab9-b5bd-b7bdb0aa9e37 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:58.674315 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af05a2f-8955-4ab9-b5bd-b7bdb0aa9e37 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:58.674872 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af05a2f-8955-4ab9-b5bd-b7bdb0aa9e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:58.701715 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-5af05a2f-8955-4ab9-b5bd-b7bdb0aa9e37 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:20:58.703239 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-5af05a2f-8955-4ab9-b5bd-b7bdb0aa9e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:20:58.787640 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 128/257] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:20:58 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Nov 02 03:20:59.358160 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c36a585-5155-4aa9-b943-9c81eec116ef None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:59.361079 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a400f-2103-4fe7-802d-73f756b48f68 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:59.361635 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a400f-2103-4fe7-802d-73f756b48f68 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:59.362383 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1a400f-2103-4fe7-802d-73f756b48f68 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:59.364012 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 129/258] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:20:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:20:59.376988 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c36a585-5155-4aa9-b943-9c81eec116ef None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:59.377199 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b9c68-6670-44eb-b5a8-823cb8d1b220 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:59.377406 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b9c68-6670-44eb-b5a8-823cb8d1b220 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:59.377726 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677b9c68-6670-44eb-b5a8-823cb8d1b220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:59.382452 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024fa93c-09c3-40bc-864b-0fbfd384885e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:59.383096 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024fa93c-09c3-40bc-864b-0fbfd384885e admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:59.383378 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024fa93c-09c3-40bc-864b-0fbfd384885e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:59.383651 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024fa93c-09c3-40bc-864b-0fbfd384885e admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:59.404887 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024fa93c-09c3-40bc-864b-0fbfd384885e admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:20:59.409871 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-024fa93c-09c3-40bc-864b-0fbfd384885e admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 03:20:59.409871 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:20:59.409871 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:20:59.409871 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:20:59.409871 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:20:59.409871 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:20:59.409871 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:20:59.409871 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:20:59.409871 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:20:59.409871 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:20:59.409871 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:20:59.409871 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:20:59.409871 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:20:59.409871 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:20:59.409871 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:20:59.409871 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:20:59.410750 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:20:59.410750 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:20:59.410750 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:20:59.410750 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:20:59.410750 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:20:59.410750 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:20:59.410750 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:20:59.410750 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:20:59.410750 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:20:59.410750 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:20:59.410750 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:20:59.410750 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:20:59.410750 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:20:59.410750 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:20:59.410750 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:20:59.410750 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:20:59.411542 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:20:59.411542 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:20:59.411542 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:20:59.411542 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:20:59.411542 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:20:59.411542 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:20:59.411542 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:20:59.411542 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:20:59.411542 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:20:59.411542 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:20:59.411542 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 03:20:59.411542 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:20:59.412196 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 130/259] 10.222.0.26 () {56 vars in 1120 bytes} [Wed Nov 2 03:20:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 54 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:20:59.413534 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-677b9c68-6670-44eb-b5a8-823cb8d1b220 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:20:59.414800 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-677b9c68-6670-44eb-b5a8-823cb8d1b220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:20:59.417062 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024fa93c-09c3-40bc-864b-0fbfd384885e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:59.434497 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 130/260] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:20:59 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Nov 02 03:20:59.434803 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024fa93c-09c3-40bc-864b-0fbfd384885e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:59.445506 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff44a913-158a-4b73-8746-8959637fdd82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:59.446123 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff44a913-158a-4b73-8746-8959637fdd82 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:59.446397 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff44a913-158a-4b73-8746-8959637fdd82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:59.446662 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff44a913-158a-4b73-8746-8959637fdd82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:59.447329 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff44a913-158a-4b73-8746-8959637fdd82 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:20:59.447685 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff44a913-158a-4b73-8746-8959637fdd82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:59.449491 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: WARNING py.warnings [None req-ff44a913-158a-4b73-8746-8959637fdd82 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Nov 02 03:20:59.449491 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: warnings.warn(msg) Nov 02 03:20:59.449491 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: Nov 02 03:20:59.450406 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff44a913-158a-4b73-8746-8959637fdd82 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:59.467105 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 131/261] 10.222.0.26 () {56 vars in 1122 bytes} [Wed Nov 2 03:20:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:20:59.472983 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677b9c68-6670-44eb-b5a8-823cb8d1b220 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:59.490140 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677b9c68-6670-44eb-b5a8-823cb8d1b220 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:59.498453 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e447c9d8-645d-463b-86a9-f894c67e4da8 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:59.498887 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e447c9d8-645d-463b-86a9-f894c67e4da8 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:59.499172 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e447c9d8-645d-463b-86a9-f894c67e4da8 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:59.499718 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c1ea31-e2be-447b-8a13-334d51fd6fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:59.500640 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c1ea31-e2be-447b-8a13-334d51fd6fcf admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:59.500836 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 132/262] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:20:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 03:20:59.501317 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c1ea31-e2be-447b-8a13-334d51fd6fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:59.501801 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c1ea31-e2be-447b-8a13-334d51fd6fcf admin admin] PATH_INFO: `/v3/groups/25665f5478654daabf86aaca12c2ab50` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:59.505705 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b2d1a-a5d0-4548-8768-73790682cef6 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:59.505991 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b2d1a-a5d0-4548-8768-73790682cef6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:59.506265 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b2d1a-a5d0-4548-8768-73790682cef6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:59.513399 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c1ea31-e2be-447b-8a13-334d51fd6fcf admin admin] RBAC: Authorizing `identity:get_group(group_id=25665f5478654daabf86aaca12c2ab50)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:59.515527 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: WARNING py.warnings [None req-c1c1ea31-e2be-447b-8a13-334d51fd6fcf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 03:20:59.515527 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: warnings.warn(msg) Nov 02 03:20:59.515527 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: Nov 02 03:20:59.517120 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c1ea31-e2be-447b-8a13-334d51fd6fcf admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:59.519276 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 131/263] 10.222.0.26 () {56 vars in 1201 bytes} [Wed Nov 2 03:20:59 2022] GET /identity/v3/groups/25665f5478654daabf86aaca12c2ab50 => generated 226 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:59.524998 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c1ea31-e2be-447b-8a13-334d51fd6fcf admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:59.539981 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c1ea31-e2be-447b-8a13-334d51fd6fcf admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:59.544427 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-b95b2d1a-a5d0-4548-8768-73790682cef6 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:20:59.545635 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b95b2d1a-a5d0-4548-8768-73790682cef6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:20:59.552982 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5021f0-d05d-4210-956e-3f764eaef948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:59.553632 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5021f0-d05d-4210-956e-3f764eaef948 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:59.553913 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5021f0-d05d-4210-956e-3f764eaef948 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:59.554188 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5021f0-d05d-4210-956e-3f764eaef948 admin admin] PATH_INFO: `/v3/projects/958251514b4b46ef908d92c563878b13` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:59.556541 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5021f0-d05d-4210-956e-3f764eaef948 admin admin] RBAC: Authorizing `identity:get_project(project_id=958251514b4b46ef908d92c563878b13)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:59.558612 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5021f0-d05d-4210-956e-3f764eaef948 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:59.562946 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 133/265] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:20:59 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Nov 02 03:20:59.563071 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 132/265] 10.222.0.26 () {56 vars in 1207 bytes} [Wed Nov 2 03:20:59 2022] GET /identity/v3/projects/958251514b4b46ef908d92c563878b13 => generated 298 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:20:59.566685 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b2d1a-a5d0-4548-8768-73790682cef6 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:59.580855 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b2d1a-a5d0-4548-8768-73790682cef6 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:59.589512 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccc347a-5e02-47b6-91ea-c6e5d6c343eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:59.590187 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc347a-5e02-47b6-91ea-c6e5d6c343eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:59.590462 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc347a-5e02-47b6-91ea-c6e5d6c343eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:59.590731 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccc347a-5e02-47b6-91ea-c6e5d6c343eb admin admin] PATH_INFO: `/v3/projects/958251514b4b46ef908d92c563878b13/groups/25665f5478654daabf86aaca12c2ab50/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:59.592672 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccc347a-5e02-47b6-91ea-c6e5d6c343eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=958251514b4b46ef908d92c563878b13, group_id=25665f5478654daabf86aaca12c2ab50, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:20:59.593697 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: WARNING py.warnings [None req-6ccc347a-5e02-47b6-91ea-c6e5d6c343eb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 03:20:59.593697 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: warnings.warn(msg) Nov 02 03:20:59.593697 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: Nov 02 03:20:59.595643 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccc347a-5e02-47b6-91ea-c6e5d6c343eb admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:20:59.622975 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 134/266] 10.222.0.26 () {58 vars in 1463 bytes} [Wed Nov 2 03:20:59 2022] PUT /identity/v3/projects/958251514b4b46ef908d92c563878b13/groups/25665f5478654daabf86aaca12c2ab50/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 03:20:59.870595 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5021f0-d05d-4210-956e-3f764eaef948 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:59.881554 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5021f0-d05d-4210-956e-3f764eaef948 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:20:59.921083 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:20:59.956761 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d37066-c19c-4c50-ba72-5d2682bed4a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:20:59.957485 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d37066-c19c-4c50-ba72-5d2682bed4a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:20:59.957768 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d37066-c19c-4c50-ba72-5d2682bed4a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:20:59.958040 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d37066-c19c-4c50-ba72-5d2682bed4a8 admin admin] PATH_INFO: `/v3/roles/member` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:20:59.970066 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d37066-c19c-4c50-ba72-5d2682bed4a8 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:20:59.975516 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-04d37066-c19c-4c50-ba72-5d2682bed4a8 admin admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 03:20:59.975516 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:20:59.975516 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:20:59.975516 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:20:59.975516 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:20:59.975516 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:20:59.975516 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:20:59.975516 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:20:59.975516 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:20:59.975516 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:20:59.975516 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:20:59.975516 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:20:59.975516 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:20:59.975516 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:20:59.975516 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:20:59.975516 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:20:59.977030 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:20:59.977030 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:20:59.977030 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:20:59.977030 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:20:59.977030 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:20:59.977030 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:20:59.977030 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:20:59.977030 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:20:59.977030 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:20:59.977030 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:20:59.977030 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:20:59.977030 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:20:59.977030 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:20:59.977030 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:20:59.977030 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:20:59.977030 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:20:59.977826 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:20:59.977826 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:20:59.977826 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:20:59.977826 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:20:59.977826 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:20:59.977826 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:20:59.977826 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:20:59.977826 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:20:59.977826 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:20:59.977826 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:20:59.977826 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 03:20:59.977826 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:20:59.977826 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 133/267] 10.222.0.26 () {56 vars in 1120 bytes} [Wed Nov 2 03:20:59 2022] GET /identity/v3/roles/member => generated 84 bytes in 107 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:20:59.984868 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccc347a-5e02-47b6-91ea-c6e5d6c343eb admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:20:59.995818 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccc347a-5e02-47b6-91ea-c6e5d6c343eb admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:00.004795 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376b7219-9a4a-45a4-a3b9-9f680b7fcf33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:00.005419 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376b7219-9a4a-45a4-a3b9-9f680b7fcf33 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:00.005699 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376b7219-9a4a-45a4-a3b9-9f680b7fcf33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:00.005966 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376b7219-9a4a-45a4-a3b9-9f680b7fcf33 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:00.008185 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d37066-c19c-4c50-ba72-5d2682bed4a8 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:00.013248 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376b7219-9a4a-45a4-a3b9-9f680b7fcf33 admin admin] RBAC: Adding query filter params (name=member) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:00.013577 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376b7219-9a4a-45a4-a3b9-9f680b7fcf33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:00.015205 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376b7219-9a4a-45a4-a3b9-9f680b7fcf33 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:00.021882 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d37066-c19c-4c50-ba72-5d2682bed4a8 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:00.031433 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8382a1a2-8809-4cda-a9fb-ce3c0e037598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:00.032101 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8382a1a2-8809-4cda-a9fb-ce3c0e037598 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:00.032375 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8382a1a2-8809-4cda-a9fb-ce3c0e037598 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:00.032639 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8382a1a2-8809-4cda-a9fb-ce3c0e037598 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:00.035161 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 135/268] 10.222.0.26 () {56 vars in 1122 bytes} [Wed Nov 2 03:20:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:00.044345 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376b7219-9a4a-45a4-a3b9-9f680b7fcf33 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:00.059231 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376b7219-9a4a-45a4-a3b9-9f680b7fcf33 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:00.060376 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8382a1a2-8809-4cda-a9fb-ce3c0e037598 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:21:00.065356 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-8382a1a2-8809-4cda-a9fb-ce3c0e037598 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 03:21:00.065356 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:00.065356 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:00.065356 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:00.065356 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:00.065356 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:00.065356 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:00.065356 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:00.065356 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:00.065356 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:00.065356 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:00.065356 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:00.065356 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:21:00.065356 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:21:00.065356 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:21:00.065356 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:21:00.067487 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:21:00.067487 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:21:00.067487 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:21:00.067487 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:21:00.067487 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:00.067487 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:00.067487 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:00.067487 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:00.067487 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:00.067487 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:00.067487 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:00.067487 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:00.067487 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:00.067487 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:00.067487 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:00.067487 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:00.069434 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:00.069434 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:00.069434 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:00.069434 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:00.069434 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:21:00.069434 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:21:00.069434 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:21:00.069434 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:21:00.069434 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:21:00.069434 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:21:00.069434 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 03:21:00.069434 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:00.069434 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 134/269] 10.222.0.26 () {56 vars in 1135 bytes} [Wed Nov 2 03:21:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 60 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 03:21:00.074088 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa9cf18-36bc-4922-8d66-ef36d06d5158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:00.074508 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa9cf18-36bc-4922-8d66-ef36d06d5158 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:00.074782 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa9cf18-36bc-4922-8d66-ef36d06d5158 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:00.075052 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa9cf18-36bc-4922-8d66-ef36d06d5158 admin admin] PATH_INFO: `/v3/groups/25665f5478654daabf86aaca12c2ab50` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:00.076944 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8382a1a2-8809-4cda-a9fb-ce3c0e037598 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:00.077934 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa9cf18-36bc-4922-8d66-ef36d06d5158 admin admin] RBAC: Authorizing `identity:get_group(group_id=25665f5478654daabf86aaca12c2ab50)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:00.078953 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: WARNING py.warnings [None req-5aa9cf18-36bc-4922-8d66-ef36d06d5158 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:get_group": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.group.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 03:21:00.078953 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: warnings.warn(msg) Nov 02 03:21:00.078953 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: Nov 02 03:21:00.080619 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa9cf18-36bc-4922-8d66-ef36d06d5158 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:00.082328 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 136/270] 10.222.0.26 () {56 vars in 1201 bytes} [Wed Nov 2 03:21:00 2022] GET /identity/v3/groups/25665f5478654daabf86aaca12c2ab50 => generated 226 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:00.089361 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa9cf18-36bc-4922-8d66-ef36d06d5158 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:00.100212 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8382a1a2-8809-4cda-a9fb-ce3c0e037598 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:00.107469 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa9cf18-36bc-4922-8d66-ef36d06d5158 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:00.110134 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcadbe1-adb8-4a2d-aad0-44fa543c02e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:00.110738 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcadbe1-adb8-4a2d-aad0-44fa543c02e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:00.111009 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcadbe1-adb8-4a2d-aad0-44fa543c02e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:00.111275 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcadbe1-adb8-4a2d-aad0-44fa543c02e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:00.111940 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcadbe1-adb8-4a2d-aad0-44fa543c02e0 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:00.112529 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcadbe1-adb8-4a2d-aad0-44fa543c02e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:00.113540 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: WARNING py.warnings [None req-9fcadbe1-adb8-4a2d-aad0-44fa543c02e0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Nov 02 03:21:00.113540 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: warnings.warn(msg) Nov 02 03:21:00.113540 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: Nov 02 03:21:00.114451 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcadbe1-adb8-4a2d-aad0-44fa543c02e0 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:00.119577 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19284453-675d-4214-af32-559c9b1b0fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:00.120251 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19284453-675d-4214-af32-559c9b1b0fed admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:00.120564 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19284453-675d-4214-af32-559c9b1b0fed admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:00.120793 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19284453-675d-4214-af32-559c9b1b0fed admin admin] PATH_INFO: `/v3/projects/953c751045ff4e78a8125850cf1d07fd` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:00.122787 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19284453-675d-4214-af32-559c9b1b0fed admin admin] RBAC: Authorizing `identity:get_project(project_id=953c751045ff4e78a8125850cf1d07fd)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:00.125504 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19284453-675d-4214-af32-559c9b1b0fed admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:00.126500 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 137/271] 10.222.0.26 () {56 vars in 1207 bytes} [Wed Nov 2 03:21:00 2022] GET /identity/v3/projects/953c751045ff4e78a8125850cf1d07fd => generated 302 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:00.131515 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 135/272] 10.222.0.26 () {56 vars in 1132 bytes} [Wed Nov 2 03:21:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:00.132645 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19284453-675d-4214-af32-559c9b1b0fed admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:00.136298 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcadbe1-adb8-4a2d-aad0-44fa543c02e0 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:00.154765 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19284453-675d-4214-af32-559c9b1b0fed admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:00.156069 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcadbe1-adb8-4a2d-aad0-44fa543c02e0 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:00.164548 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eac39e8-150b-403e-820d-67c863a3c12b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:00.165231 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d73a76-2fea-4350-b5c2-6fb46b65019d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:00.165984 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d73a76-2fea-4350-b5c2-6fb46b65019d admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:00.166259 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d73a76-2fea-4350-b5c2-6fb46b65019d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:00.166570 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d73a76-2fea-4350-b5c2-6fb46b65019d admin admin] PATH_INFO: `/v3/projects/953c751045ff4e78a8125850cf1d07fd/groups/25665f5478654daabf86aaca12c2ab50/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:00.167184 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac39e8-150b-403e-820d-67c863a3c12b admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:00.167467 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac39e8-150b-403e-820d-67c863a3c12b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:00.167737 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac39e8-150b-403e-820d-67c863a3c12b admin admin] PATH_INFO: `/v3/groups/25665f5478654daabf86aaca12c2ab50` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:00.169318 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eac39e8-150b-403e-820d-67c863a3c12b admin admin] RBAC: Authorizing `identity:get_group(group_id=25665f5478654daabf86aaca12c2ab50)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:00.170790 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d73a76-2fea-4350-b5c2-6fb46b65019d admin admin] RBAC: Authorizing `identity:create_grant(project_id=953c751045ff4e78a8125850cf1d07fd, group_id=25665f5478654daabf86aaca12c2ab50, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:00.173862 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eac39e8-150b-403e-820d-67c863a3c12b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:00.175682 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 136/273] 10.222.0.26 () {56 vars in 1201 bytes} [Wed Nov 2 03:21:00 2022] GET /identity/v3/groups/25665f5478654daabf86aaca12c2ab50 => generated 226 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:00.177154 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d73a76-2fea-4350-b5c2-6fb46b65019d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:00.181920 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eac39e8-150b-403e-820d-67c863a3c12b admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:00.202169 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 138/274] 10.222.0.26 () {58 vars in 1463 bytes} [Wed Nov 2 03:21:00 2022] PUT /identity/v3/projects/953c751045ff4e78a8125850cf1d07fd/groups/25665f5478654daabf86aaca12c2ab50/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 03:21:00.202367 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eac39e8-150b-403e-820d-67c863a3c12b admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:00.269897 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb8630b-f3eb-407b-a2c0-b759d64c0e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:00.270554 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb8630b-f3eb-407b-a2c0-b759d64c0e6e admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:00.270832 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb8630b-f3eb-407b-a2c0-b759d64c0e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:00.271106 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb8630b-f3eb-407b-a2c0-b759d64c0e6e admin admin] PATH_INFO: `/v3/projects/958251514b4b46ef908d92c563878b13` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:00.272405 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb8630b-f3eb-407b-a2c0-b759d64c0e6e admin admin] RBAC: Authorizing `identity:get_project(project_id=958251514b4b46ef908d92c563878b13)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:00.275246 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb8630b-f3eb-407b-a2c0-b759d64c0e6e admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:00.277477 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 137/275] 10.222.0.26 () {56 vars in 1207 bytes} [Wed Nov 2 03:21:00 2022] GET /identity/v3/projects/958251514b4b46ef908d92c563878b13 => generated 298 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:00.285378 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d73a76-2fea-4350-b5c2-6fb46b65019d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:00.296910 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d73a76-2fea-4350-b5c2-6fb46b65019d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:00.301773 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a8605d-2aec-4229-9e92-a40fea972ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:00.302458 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8605d-2aec-4229-9e92-a40fea972ad4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:00.302792 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8605d-2aec-4229-9e92-a40fea972ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:00.303066 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a8605d-2aec-4229-9e92-a40fea972ad4 admin admin] PATH_INFO: `/v3/projects/958251514b4b46ef908d92c563878b13/groups/25665f5478654daabf86aaca12c2ab50/roles/f25d67f31fee4a92b3060e45a3fbe379` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:00.306376 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a8605d-2aec-4229-9e92-a40fea972ad4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=958251514b4b46ef908d92c563878b13, group_id=25665f5478654daabf86aaca12c2ab50, role_id=f25d67f31fee4a92b3060e45a3fbe379)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:00.309379 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a8605d-2aec-4229-9e92-a40fea972ad4 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:00.337001 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 139/276] 10.222.0.26 () {58 vars in 1463 bytes} [Wed Nov 2 03:21:00 2022] PUT /identity/v3/projects/958251514b4b46ef908d92c563878b13/groups/25665f5478654daabf86aaca12c2ab50/roles/f25d67f31fee4a92b3060e45a3fbe379 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:21:00.546021 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34925ae4-f233-48f9-9156-c0d7f57b4d9b None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:00.546652 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34925ae4-f233-48f9-9156-c0d7f57b4d9b None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:00.547190 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34925ae4-f233-48f9-9156-c0d7f57b4d9b None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:00.549109 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 138/277] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:21:00.555157 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae896579-1088-4f95-8405-9e5f7f18d9a0 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:00.555726 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae896579-1088-4f95-8405-9e5f7f18d9a0 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:00.556225 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae896579-1088-4f95-8405-9e5f7f18d9a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:00.580961 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-ae896579-1088-4f95-8405-9e5f7f18d9a0 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:00.582619 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ae896579-1088-4f95-8405-9e5f7f18d9a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:00.638690 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:00.661157 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 140/278] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:00 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Nov 02 03:21:00.717066 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd5c533-cda0-4d8d-8048-f1571c8d7555 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:00.717674 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd5c533-cda0-4d8d-8048-f1571c8d7555 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:00.718149 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd5c533-cda0-4d8d-8048-f1571c8d7555 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:00.720021 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 139/279] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:21:00.725196 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440c146b-e988-46b8-b24b-aba5ec495fbe None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:00.725774 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440c146b-e988-46b8-b24b-aba5ec495fbe None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:00.726778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440c146b-e988-46b8-b24b-aba5ec495fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:00.748729 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-440c146b-e988-46b8-b24b-aba5ec495fbe None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:00.750281 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-440c146b-e988-46b8-b24b-aba5ec495fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:00.835047 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 141/280] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 03:21:01.086755 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd5c533-cda0-4d8d-8048-f1571c8d7555 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:01.102298 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd5c533-cda0-4d8d-8048-f1571c8d7555 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:01.107264 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c72175-406a-4e77-a34d-814a2f9f71c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:01.108144 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c72175-406a-4e77-a34d-814a2f9f71c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:01.108647 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c72175-406a-4e77-a34d-814a2f9f71c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:01.109115 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c72175-406a-4e77-a34d-814a2f9f71c0 admin admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:01.125407 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c72175-406a-4e77-a34d-814a2f9f71c0 admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:21:01.131166 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-81c72175-406a-4e77-a34d-814a2f9f71c0 admin admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 03:21:01.131166 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:01.131166 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:01.131166 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:01.131166 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:01.131166 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:01.131166 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:01.131166 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:01.131166 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:01.131166 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:01.131166 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:01.131166 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:01.131166 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:21:01.131166 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:21:01.131166 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:21:01.131166 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:21:01.132103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:21:01.132103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:21:01.132103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:21:01.132103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:21:01.132103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:01.132103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:01.132103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:01.132103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:01.132103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:01.132103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:01.132103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:01.132103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:01.132103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:01.132103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:01.132103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:01.132103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:01.132953 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:01.132953 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:01.132953 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:01.132953 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:01.132953 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:21:01.132953 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:21:01.132953 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:21:01.132953 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:21:01.132953 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:21:01.132953 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:21:01.132953 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 03:21:01.132953 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:01.135298 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 140/281] 10.222.0.26 () {56 vars in 1135 bytes} [Wed Nov 2 03:21:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 50 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:01.141293 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440c146b-e988-46b8-b24b-aba5ec495fbe None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:01.155700 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440c146b-e988-46b8-b24b-aba5ec495fbe None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:01.160455 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549f03c4-8821-4c27-92f3-bd6db5385311 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:01.161292 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549f03c4-8821-4c27-92f3-bd6db5385311 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:01.161788 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549f03c4-8821-4c27-92f3-bd6db5385311 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:01.162255 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549f03c4-8821-4c27-92f3-bd6db5385311 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:01.163140 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549f03c4-8821-4c27-92f3-bd6db5385311 admin admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:01.163711 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549f03c4-8821-4c27-92f3-bd6db5385311 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:01.165531 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549f03c4-8821-4c27-92f3-bd6db5385311 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:01.177860 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 142/282] 10.222.0.26 () {56 vars in 1132 bytes} [Wed Nov 2 03:21:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:01.183830 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c72175-406a-4e77-a34d-814a2f9f71c0 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:01.194787 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c72175-406a-4e77-a34d-814a2f9f71c0 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:01.199677 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55becaa8-8ee7-49b4-b355-590066ad4e75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:01.200651 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55becaa8-8ee7-49b4-b355-590066ad4e75 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:01.201368 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55becaa8-8ee7-49b4-b355-590066ad4e75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:01.201878 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55becaa8-8ee7-49b4-b355-590066ad4e75 admin admin] PATH_INFO: `/v3/groups/25665f5478654daabf86aaca12c2ab50` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:01.203220 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55becaa8-8ee7-49b4-b355-590066ad4e75 admin admin] RBAC: Authorizing `identity:get_group(group_id=25665f5478654daabf86aaca12c2ab50)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:01.205561 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55becaa8-8ee7-49b4-b355-590066ad4e75 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:01.207518 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 141/283] 10.222.0.26 () {56 vars in 1201 bytes} [Wed Nov 2 03:21:01 2022] GET /identity/v3/groups/25665f5478654daabf86aaca12c2ab50 => generated 226 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:01.213701 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549f03c4-8821-4c27-92f3-bd6db5385311 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:01.224612 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549f03c4-8821-4c27-92f3-bd6db5385311 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:01.229432 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310b5b88-e0f3-49cd-a863-dde76ac9d5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:01.230298 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310b5b88-e0f3-49cd-a863-dde76ac9d5da admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:01.230829 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310b5b88-e0f3-49cd-a863-dde76ac9d5da admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:01.231392 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310b5b88-e0f3-49cd-a863-dde76ac9d5da admin admin] PATH_INFO: `/v3/projects/953c751045ff4e78a8125850cf1d07fd` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:01.232748 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310b5b88-e0f3-49cd-a863-dde76ac9d5da admin admin] RBAC: Authorizing `identity:get_project(project_id=953c751045ff4e78a8125850cf1d07fd)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:01.235048 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310b5b88-e0f3-49cd-a863-dde76ac9d5da admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:01.236990 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 143/284] 10.222.0.26 () {56 vars in 1207 bytes} [Wed Nov 2 03:21:01 2022] GET /identity/v3/projects/953c751045ff4e78a8125850cf1d07fd => generated 302 bytes in 24 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:01.243383 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55becaa8-8ee7-49b4-b355-590066ad4e75 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:01.254342 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55becaa8-8ee7-49b4-b355-590066ad4e75 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:01.259845 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d7b382-ebed-4443-882f-296e57d885e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:01.262898 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d7b382-ebed-4443-882f-296e57d885e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:01.262898 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d7b382-ebed-4443-882f-296e57d885e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:01.262898 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d7b382-ebed-4443-882f-296e57d885e0 admin admin] PATH_INFO: `/v3/projects/953c751045ff4e78a8125850cf1d07fd/groups/25665f5478654daabf86aaca12c2ab50/roles/f25d67f31fee4a92b3060e45a3fbe379` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:01.263164 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310b5b88-e0f3-49cd-a863-dde76ac9d5da admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:01.269287 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d7b382-ebed-4443-882f-296e57d885e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=953c751045ff4e78a8125850cf1d07fd, group_id=25665f5478654daabf86aaca12c2ab50, role_id=f25d67f31fee4a92b3060e45a3fbe379)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:01.270554 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: WARNING py.warnings [None req-42d7b382-ebed-4443-882f-296e57d885e0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 03:21:01.270554 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: warnings.warn(msg) Nov 02 03:21:01.270554 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: Nov 02 03:21:01.272969 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d7b382-ebed-4443-882f-296e57d885e0 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:01.298999 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 142/285] 10.222.0.26 () {58 vars in 1463 bytes} [Wed Nov 2 03:21:01 2022] PUT /identity/v3/projects/953c751045ff4e78a8125850cf1d07fd/groups/25665f5478654daabf86aaca12c2ab50/roles/f25d67f31fee4a92b3060e45a3fbe379 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:21:01.405701 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310b5b88-e0f3-49cd-a863-dde76ac9d5da admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:01.426854 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:01.452830 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b16b3e-afdc-4993-a634-279ab682ca02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:01.453488 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b16b3e-afdc-4993-a634-279ab682ca02 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:01.453814 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b16b3e-afdc-4993-a634-279ab682ca02 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:01.454091 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b16b3e-afdc-4993-a634-279ab682ca02 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:01.470132 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b16b3e-afdc-4993-a634-279ab682ca02 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:21:01.475753 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-b6b16b3e-afdc-4993-a634-279ab682ca02 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 03:21:01.475753 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:01.475753 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:01.475753 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:01.475753 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:01.475753 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:01.475753 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:01.475753 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:01.475753 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:01.475753 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:01.475753 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:01.475753 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:01.475753 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:21:01.475753 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:21:01.475753 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:21:01.475753 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:21:01.477273 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:21:01.477273 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:21:01.477273 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:21:01.477273 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:21:01.477273 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:01.477273 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:01.477273 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:01.477273 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:01.477273 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:01.477273 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:01.477273 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:01.477273 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:01.477273 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:01.477273 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:01.477273 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:01.477273 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:01.478074 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:01.478074 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:01.478074 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:01.478074 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:01.478074 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:21:01.478074 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:21:01.478074 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:21:01.478074 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:21:01.478074 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:21:01.478074 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:21:01.478074 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Nov 02 03:21:01.478074 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:01.478074 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 144/286] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Nov 2 03:21:01 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:01.485232 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d7b382-ebed-4443-882f-296e57d885e0 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:01.605592 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:01.695994 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d7b382-ebed-4443-882f-296e57d885e0 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:01.743472 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bac7c2f-009b-4772-be0c-9d948ff84048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:01.744480 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac7c2f-009b-4772-be0c-9d948ff84048 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:01.744992 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac7c2f-009b-4772-be0c-9d948ff84048 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:01.745470 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bac7c2f-009b-4772-be0c-9d948ff84048 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:01.746354 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bac7c2f-009b-4772-be0c-9d948ff84048 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:01.746860 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bac7c2f-009b-4772-be0c-9d948ff84048 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:01.748502 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bac7c2f-009b-4772-be0c-9d948ff84048 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:01.764225 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 143/287] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Nov 2 03:21:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:01.770225 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b16b3e-afdc-4993-a634-279ab682ca02 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:01.813938 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5054dd86-1beb-4b4e-b8a8-5515694698af None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:01.814589 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5054dd86-1beb-4b4e-b8a8-5515694698af None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:01.815068 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5054dd86-1beb-4b4e-b8a8-5515694698af None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:01.816918 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 144/288] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 03:21:01.822185 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb39d32-8330-4c62-9bb9-fccc83a44a5a None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:01.822746 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb39d32-8330-4c62-9bb9-fccc83a44a5a None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:01.823243 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb39d32-8330-4c62-9bb9-fccc83a44a5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:01.853835 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-bdb39d32-8330-4c62-9bb9-fccc83a44a5a None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:01.855388 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-bdb39d32-8330-4c62-9bb9-fccc83a44a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:01.901965 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b16b3e-afdc-4993-a634-279ab682ca02 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:01.953934 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9170f0e6-707b-454c-9e9f-54b67d92ac5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:01.958378 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9170f0e6-707b-454c-9e9f-54b67d92ac5d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:01.958378 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9170f0e6-707b-454c-9e9f-54b67d92ac5d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:01.958378 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9170f0e6-707b-454c-9e9f-54b67d92ac5d None admin] PATH_INFO: `/v3/users/d79625965f1e42bb94193a10c3c06e2a` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:01.958378 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9170f0e6-707b-454c-9e9f-54b67d92ac5d None admin] RBAC: Authorizing `identity:get_user(user_id=d79625965f1e42bb94193a10c3c06e2a)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:01.958761 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9170f0e6-707b-454c-9e9f-54b67d92ac5d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:01.961175 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 145/289] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Nov 2 03:21:01 2022] GET /identity/v3/users/d79625965f1e42bb94193a10c3c06e2a => generated 293 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:01.968731 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9170f0e6-707b-454c-9e9f-54b67d92ac5d None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:01.999939 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 145/290] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:01 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Nov 02 03:21:02.024406 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:02.109966 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9170f0e6-707b-454c-9e9f-54b67d92ac5d None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:02.151513 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6508d9bc-8037-4cc5-92ae-29ce3f8d1a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:02.152145 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6508d9bc-8037-4cc5-92ae-29ce3f8d1a18 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:02.152418 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6508d9bc-8037-4cc5-92ae-29ce3f8d1a18 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:02.152688 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6508d9bc-8037-4cc5-92ae-29ce3f8d1a18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:02.153410 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6508d9bc-8037-4cc5-92ae-29ce3f8d1a18 None admin] RBAC: Adding query filter params (role.id=1966cb5a13b8421aad8823a14385cb43, user.id=d79625965f1e42bb94193a10c3c06e2a, scope.system=all) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:02.153715 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6508d9bc-8037-4cc5-92ae-29ce3f8d1a18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:02.155143 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6508d9bc-8037-4cc5-92ae-29ce3f8d1a18 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:02.171529 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 146/291] 10.222.0.26 () {56 vars in 1308 bytes} [Wed Nov 2 03:21:01 2022] GET /identity/v3/role_assignments?user.id=d79625965f1e42bb94193a10c3c06e2a&scope.system=all&role.id=1966cb5a13b8421aad8823a14385cb43 => generated 510 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:02.387847 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb39d32-8330-4c62-9bb9-fccc83a44a5a None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:02.397709 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb39d32-8330-4c62-9bb9-fccc83a44a5a None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:02.407775 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e37b2f-29e2-42f3-8772-3f223d026c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:02.408467 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e37b2f-29e2-42f3-8772-3f223d026c3a admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:02.408753 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e37b2f-29e2-42f3-8772-3f223d026c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:02.409029 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e37b2f-29e2-42f3-8772-3f223d026c3a admin admin] PATH_INFO: `/v3/roles/admin` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:02.420880 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e37b2f-29e2-42f3-8772-3f223d026c3a admin admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:21:02.425884 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-60e37b2f-29e2-42f3-8772-3f223d026c3a admin admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 03:21:02.425884 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:02.425884 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:02.425884 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:02.425884 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:02.425884 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:02.425884 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:02.425884 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:02.425884 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:02.425884 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:02.425884 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:02.425884 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:02.425884 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:21:02.425884 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:21:02.425884 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:21:02.425884 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:21:02.426778 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:21:02.426778 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:21:02.426778 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:21:02.426778 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:21:02.426778 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:02.426778 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:02.426778 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:02.426778 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:02.426778 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:02.426778 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:02.426778 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:02.426778 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:02.426778 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:02.426778 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:02.426778 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:02.426778 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:02.427609 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:02.427609 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:02.427609 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:02.427609 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:02.427609 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:21:02.427609 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:21:02.427609 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:21:02.427609 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:21:02.427609 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:21:02.427609 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:21:02.427609 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 03:21:02.427609 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:02.427609 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 146/292] 10.222.0.26 () {56 vars in 1117 bytes} [Wed Nov 2 03:21:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 40 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:02.433468 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6508d9bc-8037-4cc5-92ae-29ce3f8d1a18 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:02.453241 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6508d9bc-8037-4cc5-92ae-29ce3f8d1a18 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:02.461869 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9abf089-5de7-4dd1-8ccc-691f610ae442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:02.462516 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9abf089-5de7-4dd1-8ccc-691f610ae442 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:02.462912 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9abf089-5de7-4dd1-8ccc-691f610ae442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:02.463191 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9abf089-5de7-4dd1-8ccc-691f610ae442 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:02.463881 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9abf089-5de7-4dd1-8ccc-691f610ae442 admin admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:02.464208 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9abf089-5de7-4dd1-8ccc-691f610ae442 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:02.465880 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9abf089-5de7-4dd1-8ccc-691f610ae442 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:02.477721 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 147/293] 10.222.0.26 () {56 vars in 1120 bytes} [Wed Nov 2 03:21:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:02.483433 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e37b2f-29e2-42f3-8772-3f223d026c3a admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:02.501719 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e37b2f-29e2-42f3-8772-3f223d026c3a admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:02.512287 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2de25f-fc12-4865-9789-6aeea9ce4a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:02.512941 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2de25f-fc12-4865-9789-6aeea9ce4a3b admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:02.513219 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2de25f-fc12-4865-9789-6aeea9ce4a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:02.513497 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2de25f-fc12-4865-9789-6aeea9ce4a3b admin admin] PATH_INFO: `/v3/groups/a7530c4002db4ef4b7a83da573347759` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:02.523288 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2de25f-fc12-4865-9789-6aeea9ce4a3b admin admin] RBAC: Authorizing `identity:get_group(group_id=a7530c4002db4ef4b7a83da573347759)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:02.525251 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2de25f-fc12-4865-9789-6aeea9ce4a3b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:02.527233 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 147/294] 10.222.0.26 () {56 vars in 1201 bytes} [Wed Nov 2 03:21:02 2022] GET /identity/v3/groups/a7530c4002db4ef4b7a83da573347759 => generated 229 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:02.541326 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9abf089-5de7-4dd1-8ccc-691f610ae442 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:02.551683 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9abf089-5de7-4dd1-8ccc-691f610ae442 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:02.561954 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93258647-e239-4004-a1cf-c4593ca8c12b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:02.562597 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93258647-e239-4004-a1cf-c4593ca8c12b admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:02.562929 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93258647-e239-4004-a1cf-c4593ca8c12b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:02.563201 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93258647-e239-4004-a1cf-c4593ca8c12b admin admin] PATH_INFO: `/v3/projects/90ca3cc9c54d44feb93e6ed209b99483` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:02.564405 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93258647-e239-4004-a1cf-c4593ca8c12b admin admin] RBAC: Authorizing `identity:get_project(project_id=90ca3cc9c54d44feb93e6ed209b99483)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:02.566374 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93258647-e239-4004-a1cf-c4593ca8c12b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:02.568478 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 148/295] 10.222.0.26 () {56 vars in 1207 bytes} [Wed Nov 2 03:21:02 2022] GET /identity/v3/projects/90ca3cc9c54d44feb93e6ed209b99483 => generated 344 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:02.574236 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2de25f-fc12-4865-9789-6aeea9ce4a3b admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:02.589850 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2de25f-fc12-4865-9789-6aeea9ce4a3b admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:02.600404 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae950df-79cb-4b7b-a3de-7d57cec51992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:02.601115 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae950df-79cb-4b7b-a3de-7d57cec51992 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:02.601433 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae950df-79cb-4b7b-a3de-7d57cec51992 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:02.601758 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae950df-79cb-4b7b-a3de-7d57cec51992 admin admin] PATH_INFO: `/v3/projects/90ca3cc9c54d44feb93e6ed209b99483/groups/a7530c4002db4ef4b7a83da573347759/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:02.605224 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae950df-79cb-4b7b-a3de-7d57cec51992 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90ca3cc9c54d44feb93e6ed209b99483, group_id=a7530c4002db4ef4b7a83da573347759, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:02.608070 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae950df-79cb-4b7b-a3de-7d57cec51992 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:02.635391 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 148/296] 10.222.0.26 () {58 vars in 1463 bytes} [Wed Nov 2 03:21:02 2022] PUT /identity/v3/projects/90ca3cc9c54d44feb93e6ed209b99483/groups/a7530c4002db4ef4b7a83da573347759/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:21:03.027821 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a5155-a0a4-4c3d-a833-35f4d59c99a1 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:03.028262 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a5155-a0a4-4c3d-a833-35f4d59c99a1 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:03.028434 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233a5155-a0a4-4c3d-a833-35f4d59c99a1 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:03.029822 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 149/297] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:21:03.035759 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80598c29-f80d-4593-a908-ce08fcf83e84 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:03.036072 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80598c29-f80d-4593-a908-ce08fcf83e84 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:03.036345 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80598c29-f80d-4593-a908-ce08fcf83e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:03.058473 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-80598c29-f80d-4593-a908-ce08fcf83e84 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:03.059710 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-80598c29-f80d-4593-a908-ce08fcf83e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:03.100929 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:03.144224 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 149/298] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 03:21:03.427543 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233a5155-a0a4-4c3d-a833-35f4d59c99a1 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:03.528613 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:03.575619 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233a5155-a0a4-4c3d-a833-35f4d59c99a1 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:03.624641 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117ec8ba-490d-4c0c-a847-c33698fceff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:03.625656 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117ec8ba-490d-4c0c-a847-c33698fceff1 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:03.626141 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117ec8ba-490d-4c0c-a847-c33698fceff1 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:03.626616 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117ec8ba-490d-4c0c-a847-c33698fceff1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:03.643640 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117ec8ba-490d-4c0c-a847-c33698fceff1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:21:03.648984 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-117ec8ba-490d-4c0c-a847-c33698fceff1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 03:21:03.648984 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:03.648984 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:03.648984 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:03.648984 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:03.648984 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:03.648984 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:03.648984 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:03.648984 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:03.648984 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:03.648984 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:03.648984 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:03.648984 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:21:03.648984 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:21:03.648984 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:21:03.648984 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:21:03.649858 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:21:03.649858 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:21:03.649858 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:21:03.649858 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:21:03.649858 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:03.649858 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:03.649858 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:03.649858 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:03.649858 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:03.649858 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:03.649858 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:03.649858 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:03.649858 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:03.649858 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:03.649858 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:03.649858 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:03.650660 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:03.650660 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:03.650660 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:03.650660 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:03.650660 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:21:03.650660 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:21:03.650660 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:21:03.650660 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:21:03.650660 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:21:03.650660 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:21:03.650660 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 03:21:03.650660 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:03.653195 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 150/299] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Nov 2 03:21:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:03.659153 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80598c29-f80d-4593-a908-ce08fcf83e84 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:03.802620 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80598c29-f80d-4593-a908-ce08fcf83e84 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:03.846831 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c522793e-a683-4c97-8d6b-b98a06958bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:03.847447 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c522793e-a683-4c97-8d6b-b98a06958bec None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:03.847722 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c522793e-a683-4c97-8d6b-b98a06958bec None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:03.848008 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c522793e-a683-4c97-8d6b-b98a06958bec None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:03.848679 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c522793e-a683-4c97-8d6b-b98a06958bec None admin] RBAC: Adding query filter params (name=member) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:03.848975 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c522793e-a683-4c97-8d6b-b98a06958bec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:03.850335 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c522793e-a683-4c97-8d6b-b98a06958bec None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:03.864061 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 150/300] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Nov 2 03:21:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:03.865901 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117ec8ba-490d-4c0c-a847-c33698fceff1 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:03.993679 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d92cb3f-598d-45b2-8a22-5d12ae9928a3 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:03.993989 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d92cb3f-598d-45b2-8a22-5d12ae9928a3 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:03.994265 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d92cb3f-598d-45b2-8a22-5d12ae9928a3 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:03.995610 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 151/301] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 03:21:04.005284 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc58363-3d87-439c-8ffc-2d1b0a331cf0 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:04.005572 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc58363-3d87-439c-8ffc-2d1b0a331cf0 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:04.005845 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc58363-3d87-439c-8ffc-2d1b0a331cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:04.024208 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117ec8ba-490d-4c0c-a847-c33698fceff1 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:04.035227 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-6fc58363-3d87-439c-8ffc-2d1b0a331cf0 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:04.036561 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-6fc58363-3d87-439c-8ffc-2d1b0a331cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:04.061197 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5882abf-9694-4635-9d38-acec05b57156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:04.062154 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5882abf-9694-4635-9d38-acec05b57156 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:04.062638 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5882abf-9694-4635-9d38-acec05b57156 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:04.063118 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5882abf-9694-4635-9d38-acec05b57156 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:04.066076 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:04.077263 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:04.136857 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 152/302] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 03:21:04.138300 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce065a-26ab-45e4-a0a1-ee448d0c353a None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:04.138594 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce065a-26ab-45e4-a0a1-ee448d0c353a None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:04.138873 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ce065a-26ab-45e4-a0a1-ee448d0c353a None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:04.142847 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 153/303] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:21:04.150215 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe3230-a434-4cf5-9a96-a37ada3e03b4 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:04.150493 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe3230-a434-4cf5-9a96-a37ada3e03b4 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:04.150764 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fe3230-a434-4cf5-9a96-a37ada3e03b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:04.154260 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5882abf-9694-4635-9d38-acec05b57156 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:04.155704 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5882abf-9694-4635-9d38-acec05b57156 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:04.182251 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-22fe3230-a434-4cf5-9a96-a37ada3e03b4 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:04.183434 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-22fe3230-a434-4cf5-9a96-a37ada3e03b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:04.297311 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 154/304] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 03:21:04.322387 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 151/305] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Nov 2 03:21:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 03:21:04.328195 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fe3230-a434-4cf5-9a96-a37ada3e03b4 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:04.442849 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:04.458149 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fe3230-a434-4cf5-9a96-a37ada3e03b4 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:04.478725 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5882abf-9694-4635-9d38-acec05b57156 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:04.512132 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d23150-b846-4163-b84d-372672447116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:04.512783 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d23150-b846-4163-b84d-372672447116 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:04.513068 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d23150-b846-4163-b84d-372672447116 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:04.513347 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d23150-b846-4163-b84d-372672447116 None admin] PATH_INFO: `/v3/projects/958251514b4b46ef908d92c563878b13` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:04.514616 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d23150-b846-4163-b84d-372672447116 None admin] RBAC: Authorizing `identity:get_project(project_id=958251514b4b46ef908d92c563878b13)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:04.516238 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d23150-b846-4163-b84d-372672447116 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:04.517945 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 155/306] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Nov 2 03:21:04 2022] GET /identity/v3/projects/958251514b4b46ef908d92c563878b13 => generated 298 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:04.523930 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d23150-b846-4163-b84d-372672447116 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:04.554634 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:04.686814 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5882abf-9694-4635-9d38-acec05b57156 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:04.713299 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d23150-b846-4163-b84d-372672447116 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:04.732988 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f350c-36bc-49d0-8a77-bbf09d3000ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:04.733632 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f350c-36bc-49d0-8a77-bbf09d3000ae None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:04.733915 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f350c-36bc-49d0-8a77-bbf09d3000ae None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:04.734191 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f350c-36bc-49d0-8a77-bbf09d3000ae None admin] PATH_INFO: `/v3/roles/member` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:04.747763 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00f350c-36bc-49d0-8a77-bbf09d3000ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:21:04.751532 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525eb7fe-89a9-48ca-80de-5184f011bc00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:04.752195 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525eb7fe-89a9-48ca-80de-5184f011bc00 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:04.752476 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525eb7fe-89a9-48ca-80de-5184f011bc00 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:04.752746 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525eb7fe-89a9-48ca-80de-5184f011bc00 None admin] PATH_INFO: `/v3/groups/25665f5478654daabf86aaca12c2ab50` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:04.753540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-e00f350c-36bc-49d0-8a77-bbf09d3000ae None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Nov 02 03:21:04.753540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:04.753540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:04.753540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:04.753540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:04.753540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:04.753540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:04.753540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:04.753540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:04.753540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:04.753540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:04.753540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:04.753540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:21:04.753540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:21:04.753540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:21:04.753540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:21:04.754416 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:21:04.754416 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:21:04.754416 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:21:04.754416 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:21:04.754416 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:04.754416 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:04.754416 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:04.754416 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:04.754416 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:04.754416 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:04.754416 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:04.754416 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:04.754416 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:04.754416 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:04.754416 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:04.754416 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:04.755218 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:04.755218 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:04.755218 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:04.755218 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:04.755218 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:21:04.755218 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:21:04.755218 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:21:04.755218 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:21:04.755218 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:21:04.755218 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:21:04.755218 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Nov 02 03:21:04.755218 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:04.756636 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525eb7fe-89a9-48ca-80de-5184f011bc00 None admin] RBAC: Authorizing `identity:get_group(group_id=25665f5478654daabf86aaca12c2ab50)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:04.756720 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 152/307] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Nov 2 03:21:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:04.756788 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-525eb7fe-89a9-48ca-80de-5184f011bc00 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:04.756854 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f350c-36bc-49d0-8a77-bbf09d3000ae None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:04.756919 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 156/308] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Nov 2 03:21:04 2022] GET /identity/v3/groups/25665f5478654daabf86aaca12c2ab50 => generated 226 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:04.760823 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525eb7fe-89a9-48ca-80de-5184f011bc00 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:04.793656 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:04.878441 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525eb7fe-89a9-48ca-80de-5184f011bc00 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:04.886888 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:04.895317 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f350c-36bc-49d0-8a77-bbf09d3000ae None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:04.921580 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0481ddd-59f1-441f-88ff-6323957f7deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:04.922459 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0481ddd-59f1-441f-88ff-6323957f7deb None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:04.922739 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0481ddd-59f1-441f-88ff-6323957f7deb None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:04.923010 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0481ddd-59f1-441f-88ff-6323957f7deb None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:04.923675 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0481ddd-59f1-441f-88ff-6323957f7deb None admin] RBAC: Adding query filter params (name=member) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:04.927612 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0481ddd-59f1-441f-88ff-6323957f7deb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:04.927612 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0481ddd-59f1-441f-88ff-6323957f7deb None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:04.937716 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 157/309] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Nov 2 03:21:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:04.938476 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0481ddd-59f1-441f-88ff-6323957f7deb None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:04.943474 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3058b0c-ff96-46fd-86b6-097d1883b53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:04.944335 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3058b0c-ff96-46fd-86b6-097d1883b53f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:04.944526 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3058b0c-ff96-46fd-86b6-097d1883b53f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:04.944838 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3058b0c-ff96-46fd-86b6-097d1883b53f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:04.960803 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3058b0c-ff96-46fd-86b6-097d1883b53f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:21:04.965733 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-a3058b0c-ff96-46fd-86b6-097d1883b53f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 03:21:04.965733 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:04.965733 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:04.965733 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:04.965733 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:04.965733 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:04.965733 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:04.965733 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:04.965733 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:04.965733 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:04.965733 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:04.965733 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:04.965733 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:21:04.965733 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:21:04.965733 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:21:04.965733 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:21:04.966832 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:21:04.966832 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:21:04.966832 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:21:04.966832 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:21:04.966832 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:04.966832 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:04.966832 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:04.966832 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:04.966832 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:04.966832 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:04.966832 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:04.966832 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:04.966832 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:04.966832 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:04.966832 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:04.966832 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:04.967820 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:04.967820 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:04.967820 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:04.967820 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:04.967820 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:21:04.967820 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:21:04.967820 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:21:04.967820 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:21:04.967820 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:21:04.967820 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:21:04.967820 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 03:21:04.967820 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:04.967820 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 153/310] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Nov 2 03:21:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:04.968704 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b07a16f-bec0-4863-a149-3f46f52f4377 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:04.968704 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b07a16f-bec0-4863-a149-3f46f52f4377 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:04.968846 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b07a16f-bec0-4863-a149-3f46f52f4377 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:04.970604 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 154/311] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:21:04.972774 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b07a16f-bec0-4863-a149-3f46f52f4377 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:05.075026 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:05.085594 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0481ddd-59f1-441f-88ff-6323957f7deb None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:05.109085 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b07a16f-bec0-4863-a149-3f46f52f4377 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:05.126206 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bd7ca6-b7fa-4605-8f28-b4092de5c07c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:05.127430 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd7ca6-b7fa-4605-8f28-b4092de5c07c None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:05.127907 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd7ca6-b7fa-4605-8f28-b4092de5c07c None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:05.128511 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd7ca6-b7fa-4605-8f28-b4092de5c07c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:05.129245 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bd7ca6-b7fa-4605-8f28-b4092de5c07c None admin] RBAC: Adding query filter params (group.id=25665f5478654daabf86aaca12c2ab50, role.id=7a6df2ce66694822bce7983f7d988da5, scope.project.id=958251514b4b46ef908d92c563878b13) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:05.129555 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bd7ca6-b7fa-4605-8f28-b4092de5c07c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:05.131421 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bd7ca6-b7fa-4605-8f28-b4092de5c07c None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:05.140094 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 158/312] 10.222.0.26 () {56 vars in 1376 bytes} [Wed Nov 2 03:21:04 2022] GET /identity/v3/role_assignments?group.id=25665f5478654daabf86aaca12c2ab50&scope.project.id=958251514b4b46ef908d92c563878b13&role.id=7a6df2ce66694822bce7983f7d988da5 => generated 611 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:05.140977 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bd7ca6-b7fa-4605-8f28-b4092de5c07c None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:05.174074 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f208d3c7-4047-46fb-84f4-26c1e955b5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:05.175401 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f208d3c7-4047-46fb-84f4-26c1e955b5c4 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:05.175934 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f208d3c7-4047-46fb-84f4-26c1e955b5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:05.176661 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f208d3c7-4047-46fb-84f4-26c1e955b5c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:05.216190 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:05.276723 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bd7ca6-b7fa-4605-8f28-b4092de5c07c None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:05.286635 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f208d3c7-4047-46fb-84f4-26c1e955b5c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:05.289560 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f208d3c7-4047-46fb-84f4-26c1e955b5c4 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:05.320300 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23f1693-7d14-4462-a38b-d2fe67eb2058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:05.320940 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23f1693-7d14-4462-a38b-d2fe67eb2058 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:05.321219 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23f1693-7d14-4462-a38b-d2fe67eb2058 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:05.321492 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23f1693-7d14-4462-a38b-d2fe67eb2058 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:05.322201 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23f1693-7d14-4462-a38b-d2fe67eb2058 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:05.322506 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23f1693-7d14-4462-a38b-d2fe67eb2058 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:05.323939 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23f1693-7d14-4462-a38b-d2fe67eb2058 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:05.334791 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 159/313] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Nov 2 03:21:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:05.339990 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca37d15-6cd2-4fea-b933-b2c7655d8a39 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:05.339990 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca37d15-6cd2-4fea-b933-b2c7655d8a39 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:05.339990 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca37d15-6cd2-4fea-b933-b2c7655d8a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:05.359376 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-cca37d15-6cd2-4fea-b933-b2c7655d8a39 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:05.360625 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-cca37d15-6cd2-4fea-b933-b2c7655d8a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:05.376364 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:05.433214 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 155/314] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Nov 2 03:21:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 462 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 03:21:05.433914 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f208d3c7-4047-46fb-84f4-26c1e955b5c4 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:05.453933 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 160/315] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 03:21:05.455228 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5463c027-3560-4840-85bb-39b42c560738 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:05.455498 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5463c027-3560-4840-85bb-39b42c560738 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:05.455779 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5463c027-3560-4840-85bb-39b42c560738 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:05.457776 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 161/316] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:21:05.458826 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5463c027-3560-4840-85bb-39b42c560738 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:05.506854 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:05.563127 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f208d3c7-4047-46fb-84f4-26c1e955b5c4 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:05.595406 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5463c027-3560-4840-85bb-39b42c560738 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:05.602437 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e45ce6e-2b5c-49b5-8d8a-84eb6e137cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:05.603062 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e45ce6e-2b5c-49b5-8d8a-84eb6e137cab None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:05.603341 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e45ce6e-2b5c-49b5-8d8a-84eb6e137cab None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:05.603615 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e45ce6e-2b5c-49b5-8d8a-84eb6e137cab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:05.634156 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53dcec5-23b8-4a0c-9f90-889afb1484ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:05.634792 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53dcec5-23b8-4a0c-9f90-889afb1484ad None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:05.635074 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53dcec5-23b8-4a0c-9f90-889afb1484ad None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:05.635345 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53dcec5-23b8-4a0c-9f90-889afb1484ad None admin] PATH_INFO: `/v3/projects/953c751045ff4e78a8125850cf1d07fd` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:05.636498 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53dcec5-23b8-4a0c-9f90-889afb1484ad None admin] RBAC: Authorizing `identity:get_project(project_id=953c751045ff4e78a8125850cf1d07fd)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:05.637942 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53dcec5-23b8-4a0c-9f90-889afb1484ad None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:05.639790 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 162/317] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Nov 2 03:21:05 2022] GET /identity/v3/projects/953c751045ff4e78a8125850cf1d07fd => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:05.640728 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859392b4-574e-4312-830b-b9a1f3c2a158 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:05.641013 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859392b4-574e-4312-830b-b9a1f3c2a158 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:05.641363 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859392b4-574e-4312-830b-b9a1f3c2a158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:05.660689 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-859392b4-574e-4312-830b-b9a1f3c2a158 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:05.661892 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-859392b4-574e-4312-830b-b9a1f3c2a158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:05.672282 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:05.682409 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e45ce6e-2b5c-49b5-8d8a-84eb6e137cab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:05.684292 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e45ce6e-2b5c-49b5-8d8a-84eb6e137cab None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:05.741239 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 163/318] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 03:21:05.742472 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859392b4-574e-4312-830b-b9a1f3c2a158 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:05.746427 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:05.804244 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 156/319] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Nov 2 03:21:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 03:21:05.808830 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e45ce6e-2b5c-49b5-8d8a-84eb6e137cab None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:05.866400 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859392b4-574e-4312-830b-b9a1f3c2a158 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:05.906700 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b6621c-62a1-4e28-8cc5-d24da0008e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:05.907340 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b6621c-62a1-4e28-8cc5-d24da0008e70 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:05.907621 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b6621c-62a1-4e28-8cc5-d24da0008e70 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:05.907895 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b6621c-62a1-4e28-8cc5-d24da0008e70 None admin] PATH_INFO: `/v3/groups/25665f5478654daabf86aaca12c2ab50` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:05.909032 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b6621c-62a1-4e28-8cc5-d24da0008e70 None admin] RBAC: Authorizing `identity:get_group(group_id=25665f5478654daabf86aaca12c2ab50)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:05.910456 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b6621c-62a1-4e28-8cc5-d24da0008e70 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:05.912050 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 164/320] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Nov 2 03:21:05 2022] GET /identity/v3/groups/25665f5478654daabf86aaca12c2ab50 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:05.915997 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b6621c-62a1-4e28-8cc5-d24da0008e70 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:05.927881 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:05.934199 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e45ce6e-2b5c-49b5-8d8a-84eb6e137cab None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:05.986367 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e6c5a6-7256-4786-9e0d-114c05ac875a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:05.987002 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e6c5a6-7256-4786-9e0d-114c05ac875a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:05.987531 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e6c5a6-7256-4786-9e0d-114c05ac875a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:05.987796 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e6c5a6-7256-4786-9e0d-114c05ac875a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:06.002439 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e6c5a6-7256-4786-9e0d-114c05ac875a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:21:06.011176 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-85e6c5a6-7256-4786-9e0d-114c05ac875a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 03:21:06.011176 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:06.011176 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:06.011176 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:06.011176 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:06.011176 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:06.011176 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:06.011176 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:06.011176 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:06.011176 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:06.011176 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:06.011176 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:06.011176 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:21:06.011176 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:21:06.011176 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:21:06.011176 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:21:06.012751 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:21:06.012751 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:21:06.012751 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:21:06.012751 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:21:06.012751 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:06.012751 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:06.012751 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:06.012751 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:06.012751 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:06.012751 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:06.012751 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:06.012751 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:06.012751 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:06.012751 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:06.012751 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:06.012751 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:06.013582 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:06.013582 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:06.013582 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:06.013582 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:06.013582 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:21:06.013582 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:21:06.013582 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:21:06.013582 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:21:06.013582 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:21:06.013582 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:21:06.013582 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 03:21:06.013582 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:06.013582 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 157/321] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Nov 2 03:21:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:06.013582 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e6c5a6-7256-4786-9e0d-114c05ac875a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:06.034432 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:06.035132 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b6621c-62a1-4e28-8cc5-d24da0008e70 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:06.073744 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf84e126-1926-4f01-934f-bf0f5dd6b3b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:06.074301 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf84e126-1926-4f01-934f-bf0f5dd6b3b5 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:06.074583 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf84e126-1926-4f01-934f-bf0f5dd6b3b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:06.074856 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf84e126-1926-4f01-934f-bf0f5dd6b3b5 None admin] PATH_INFO: `/v3/projects/958251514b4b46ef908d92c563878b13` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:06.076665 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf84e126-1926-4f01-934f-bf0f5dd6b3b5 None admin] RBAC: Authorizing `identity:get_project(project_id=958251514b4b46ef908d92c563878b13)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:06.077497 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf84e126-1926-4f01-934f-bf0f5dd6b3b5 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:06.079141 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 165/322] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Nov 2 03:21:05 2022] GET /identity/v3/projects/958251514b4b46ef908d92c563878b13 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:06.079902 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf84e126-1926-4f01-934f-bf0f5dd6b3b5 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:06.133582 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e6c5a6-7256-4786-9e0d-114c05ac875a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:06.171012 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:06.174519 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003760e8-cfc7-4e3a-ba00-041fe5816053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:06.175138 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003760e8-cfc7-4e3a-ba00-041fe5816053 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:06.175417 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003760e8-cfc7-4e3a-ba00-041fe5816053 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:06.175689 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003760e8-cfc7-4e3a-ba00-041fe5816053 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:06.176452 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003760e8-cfc7-4e3a-ba00-041fe5816053 None admin] RBAC: Adding query filter params (group.id=25665f5478654daabf86aaca12c2ab50, role.id=7a6df2ce66694822bce7983f7d988da5, scope.project.id=953c751045ff4e78a8125850cf1d07fd) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:06.176765 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003760e8-cfc7-4e3a-ba00-041fe5816053 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:06.178257 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003760e8-cfc7-4e3a-ba00-041fe5816053 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:06.188831 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 158/323] 10.222.0.26 () {56 vars in 1376 bytes} [Wed Nov 2 03:21:06 2022] GET /identity/v3/role_assignments?group.id=25665f5478654daabf86aaca12c2ab50&scope.project.id=953c751045ff4e78a8125850cf1d07fd&role.id=7a6df2ce66694822bce7983f7d988da5 => generated 611 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:06.190751 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003760e8-cfc7-4e3a-ba00-041fe5816053 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:06.195643 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf84e126-1926-4f01-934f-bf0f5dd6b3b5 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:06.243375 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5454845f-17d2-4c77-8772-9f8cc2b46e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:06.244035 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5454845f-17d2-4c77-8772-9f8cc2b46e05 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:06.244321 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5454845f-17d2-4c77-8772-9f8cc2b46e05 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:06.244595 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5454845f-17d2-4c77-8772-9f8cc2b46e05 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:06.245293 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5454845f-17d2-4c77-8772-9f8cc2b46e05 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:06.245597 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5454845f-17d2-4c77-8772-9f8cc2b46e05 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:06.247027 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5454845f-17d2-4c77-8772-9f8cc2b46e05 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:06.261469 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 166/324] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Nov 2 03:21:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:06.262508 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5454845f-17d2-4c77-8772-9f8cc2b46e05 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:06.308321 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:06.341754 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003760e8-cfc7-4e3a-ba00-041fe5816053 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:06.385165 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0ff12f-1954-4f64-9171-308b6cd07758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:06.385811 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0ff12f-1954-4f64-9171-308b6cd07758 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:06.386099 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0ff12f-1954-4f64-9171-308b6cd07758 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:06.386373 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0ff12f-1954-4f64-9171-308b6cd07758 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:06.389620 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5454845f-17d2-4c77-8772-9f8cc2b46e05 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:06.397734 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0ff12f-1954-4f64-9171-308b6cd07758 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:21:06.403714 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-5e0ff12f-1954-4f64-9171-308b6cd07758 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 03:21:06.403714 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:06.403714 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:06.403714 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:06.403714 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:06.403714 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:06.403714 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:06.403714 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:06.403714 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:06.403714 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:06.403714 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:06.403714 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:06.403714 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:21:06.403714 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:21:06.403714 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:21:06.403714 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:21:06.404794 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:21:06.404794 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:21:06.404794 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:21:06.404794 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:21:06.404794 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:06.404794 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:06.404794 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:06.404794 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:06.404794 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:06.404794 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:06.404794 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:06.404794 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:06.404794 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:06.404794 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:06.404794 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:06.404794 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:06.405588 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:06.405588 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:06.405588 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:06.405588 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:06.405588 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:21:06.405588 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:21:06.405588 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:21:06.405588 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:21:06.405588 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:21:06.405588 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:21:06.405588 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 03:21:06.405588 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:06.405588 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 159/325] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Nov 2 03:21:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:06.406643 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0ff12f-1954-4f64-9171-308b6cd07758 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:06.430549 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5493cdcb-f560-444c-9038-87be79454b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:06.431178 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5493cdcb-f560-444c-9038-87be79454b36 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:06.431453 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5493cdcb-f560-444c-9038-87be79454b36 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:06.431723 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5493cdcb-f560-444c-9038-87be79454b36 None admin] PATH_INFO: `/v3/groups/25665f5478654daabf86aaca12c2ab50` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:06.432861 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5493cdcb-f560-444c-9038-87be79454b36 None admin] RBAC: Authorizing `identity:get_group(group_id=25665f5478654daabf86aaca12c2ab50)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:06.434366 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5493cdcb-f560-444c-9038-87be79454b36 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:06.435997 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 167/326] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Nov 2 03:21:06 2022] GET /identity/v3/groups/25665f5478654daabf86aaca12c2ab50 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:06.436809 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5493cdcb-f560-444c-9038-87be79454b36 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:06.454086 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:06.512957 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0ff12f-1954-4f64-9171-308b6cd07758 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:06.545942 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5493cdcb-f560-444c-9038-87be79454b36 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:06.550829 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0212a93-5f15-4d30-b398-448fd662a286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:06.551444 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0212a93-5f15-4d30-b398-448fd662a286 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:06.551726 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0212a93-5f15-4d30-b398-448fd662a286 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:06.552026 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0212a93-5f15-4d30-b398-448fd662a286 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:06.561065 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:06.579524 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf98bfe-3dac-4d7b-a14a-5119b6e50959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:06.580170 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf98bfe-3dac-4d7b-a14a-5119b6e50959 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:06.580460 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf98bfe-3dac-4d7b-a14a-5119b6e50959 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:06.580736 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf98bfe-3dac-4d7b-a14a-5119b6e50959 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:06.581404 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf98bfe-3dac-4d7b-a14a-5119b6e50959 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:06.581710 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf98bfe-3dac-4d7b-a14a-5119b6e50959 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:06.583073 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bf98bfe-3dac-4d7b-a14a-5119b6e50959 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:06.591009 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 168/327] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Nov 2 03:21:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:06.591844 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf98bfe-3dac-4d7b-a14a-5119b6e50959 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:06.626167 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0212a93-5f15-4d30-b398-448fd662a286 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:06.627635 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0212a93-5f15-4d30-b398-448fd662a286 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:06.661767 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:06.691012 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf98bfe-3dac-4d7b-a14a-5119b6e50959 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:06.724349 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711355ee-4d0e-4ee4-931c-43fd16fee0c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:06.724948 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711355ee-4d0e-4ee4-931c-43fd16fee0c9 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:06.725230 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711355ee-4d0e-4ee4-931c-43fd16fee0c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:06.725505 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711355ee-4d0e-4ee4-931c-43fd16fee0c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:06.725832 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 160/328] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Nov 2 03:21:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 03:21:06.726262 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711355ee-4d0e-4ee4-931c-43fd16fee0c9 None admin] RBAC: Adding query filter params (group.id=25665f5478654daabf86aaca12c2ab50, role.id=f25d67f31fee4a92b3060e45a3fbe379, scope.project.id=958251514b4b46ef908d92c563878b13) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:06.726570 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711355ee-4d0e-4ee4-931c-43fd16fee0c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:06.726722 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0212a93-5f15-4d30-b398-448fd662a286 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:06.728018 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711355ee-4d0e-4ee4-931c-43fd16fee0c9 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:06.740545 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 169/329] 10.222.0.26 () {56 vars in 1376 bytes} [Wed Nov 2 03:21:06 2022] GET /identity/v3/role_assignments?group.id=25665f5478654daabf86aaca12c2ab50&scope.project.id=958251514b4b46ef908d92c563878b13&role.id=f25d67f31fee4a92b3060e45a3fbe379 => generated 611 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:06.740545 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711355ee-4d0e-4ee4-931c-43fd16fee0c9 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:06.770697 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:06.841215 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0212a93-5f15-4d30-b398-448fd662a286 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:06.852544 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711355ee-4d0e-4ee4-931c-43fd16fee0c9 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:06.881010 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77366dd-6f6a-4392-ae75-099a4ee995cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:06.881945 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77366dd-6f6a-4392-ae75-099a4ee995cf None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:06.882419 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77366dd-6f6a-4392-ae75-099a4ee995cf None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:06.882906 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77366dd-6f6a-4392-ae75-099a4ee995cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:06.890081 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7457975-9599-469d-b6ae-7ac5365c3134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:06.891149 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7457975-9599-469d-b6ae-7ac5365c3134 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:06.891439 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7457975-9599-469d-b6ae-7ac5365c3134 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:06.891721 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7457975-9599-469d-b6ae-7ac5365c3134 None admin] PATH_INFO: `/v3/projects/953c751045ff4e78a8125850cf1d07fd` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:06.893500 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7457975-9599-469d-b6ae-7ac5365c3134 None admin] RBAC: Authorizing `identity:get_project(project_id=953c751045ff4e78a8125850cf1d07fd)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:06.895364 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7457975-9599-469d-b6ae-7ac5365c3134 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:06.897675 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 170/330] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Nov 2 03:21:06 2022] GET /identity/v3/projects/953c751045ff4e78a8125850cf1d07fd => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:06.903704 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7457975-9599-469d-b6ae-7ac5365c3134 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:06.907498 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:06.968903 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77366dd-6f6a-4392-ae75-099a4ee995cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:06.970350 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77366dd-6f6a-4392-ae75-099a4ee995cf None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:07.017550 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7457975-9599-469d-b6ae-7ac5365c3134 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:07.020366 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:07.056109 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad578269-dc26-44d4-a947-dbaeefb48f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:07.056751 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad578269-dc26-44d4-a947-dbaeefb48f84 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:07.057035 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad578269-dc26-44d4-a947-dbaeefb48f84 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:07.057311 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad578269-dc26-44d4-a947-dbaeefb48f84 None admin] PATH_INFO: `/v3/groups/25665f5478654daabf86aaca12c2ab50` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:07.058550 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad578269-dc26-44d4-a947-dbaeefb48f84 None admin] RBAC: Authorizing `identity:get_group(group_id=25665f5478654daabf86aaca12c2ab50)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:07.060857 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad578269-dc26-44d4-a947-dbaeefb48f84 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:07.062639 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 171/331] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Nov 2 03:21:06 2022] GET /identity/v3/groups/25665f5478654daabf86aaca12c2ab50 => generated 226 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:07.068659 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad578269-dc26-44d4-a947-dbaeefb48f84 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:07.094935 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 161/332] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Nov 2 03:21:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 03:21:07.100614 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77366dd-6f6a-4392-ae75-099a4ee995cf None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:07.150604 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:07.185822 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad578269-dc26-44d4-a947-dbaeefb48f84 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:07.220842 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77366dd-6f6a-4392-ae75-099a4ee995cf None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:07.222990 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed76f0e-7d46-4377-bba3-0fd776f92029 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:07.223609 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed76f0e-7d46-4377-bba3-0fd776f92029 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:07.223879 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed76f0e-7d46-4377-bba3-0fd776f92029 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:07.224188 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed76f0e-7d46-4377-bba3-0fd776f92029 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:07.224915 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed76f0e-7d46-4377-bba3-0fd776f92029 None admin] RBAC: Adding query filter params (group.id=25665f5478654daabf86aaca12c2ab50, role.id=f25d67f31fee4a92b3060e45a3fbe379, scope.project.id=953c751045ff4e78a8125850cf1d07fd) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:07.225221 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed76f0e-7d46-4377-bba3-0fd776f92029 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:07.227211 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed76f0e-7d46-4377-bba3-0fd776f92029 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:07.235316 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 172/333] 10.222.0.26 () {56 vars in 1376 bytes} [Wed Nov 2 03:21:07 2022] GET /identity/v3/role_assignments?group.id=25665f5478654daabf86aaca12c2ab50&scope.project.id=953c751045ff4e78a8125850cf1d07fd&role.id=f25d67f31fee4a92b3060e45a3fbe379 => generated 611 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:21:07.259619 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5c3349-d077-4d60-b55d-4736e0f83119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:07.260310 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c3349-d077-4d60-b55d-4736e0f83119 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:07.260591 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c3349-d077-4d60-b55d-4736e0f83119 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:07.261442 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5c3349-d077-4d60-b55d-4736e0f83119 None admin] PATH_INFO: `/v3/projects/90ca3cc9c54d44feb93e6ed209b99483` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:07.262074 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5c3349-d077-4d60-b55d-4736e0f83119 None admin] RBAC: Authorizing `identity:get_project(project_id=90ca3cc9c54d44feb93e6ed209b99483)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:07.263573 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5c3349-d077-4d60-b55d-4736e0f83119 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:07.265366 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 162/334] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Nov 2 03:21:07 2022] GET /identity/v3/projects/90ca3cc9c54d44feb93e6ed209b99483 => generated 344 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:21:07.271412 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed76f0e-7d46-4377-bba3-0fd776f92029 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:07.381662 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed76f0e-7d46-4377-bba3-0fd776f92029 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:07.419532 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576570e5-eac2-4006-a9df-f14762606ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:07.420135 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576570e5-eac2-4006-a9df-f14762606ca9 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:07.420417 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576570e5-eac2-4006-a9df-f14762606ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:07.420690 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576570e5-eac2-4006-a9df-f14762606ca9 None admin] PATH_INFO: `/v3/groups/a7530c4002db4ef4b7a83da573347759` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:07.421805 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576570e5-eac2-4006-a9df-f14762606ca9 None admin] RBAC: Authorizing `identity:get_group(group_id=a7530c4002db4ef4b7a83da573347759)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:07.423255 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576570e5-eac2-4006-a9df-f14762606ca9 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:07.424932 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 173/335] 10.222.0.26 () {56 vars in 1180 bytes} [Wed Nov 2 03:21:07 2022] GET /identity/v3/groups/a7530c4002db4ef4b7a83da573347759 => generated 229 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:07.431379 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5c3349-d077-4d60-b55d-4736e0f83119 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:07.440843 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:07.537017 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5c3349-d077-4d60-b55d-4736e0f83119 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:07.571575 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e87aad-389b-45a0-8e73-15fb0dc0886c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:07.572220 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e87aad-389b-45a0-8e73-15fb0dc0886c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:07.572502 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e87aad-389b-45a0-8e73-15fb0dc0886c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:07.572773 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e87aad-389b-45a0-8e73-15fb0dc0886c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:07.573496 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e87aad-389b-45a0-8e73-15fb0dc0886c None admin] RBAC: Adding query filter params (group.id=a7530c4002db4ef4b7a83da573347759, role.id=addc0db6b4ac4362b07fe21e14f89180, scope.project.id=90ca3cc9c54d44feb93e6ed209b99483) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:07.573800 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e87aad-389b-45a0-8e73-15fb0dc0886c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:07.575229 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e87aad-389b-45a0-8e73-15fb0dc0886c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:07.581095 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 163/336] 10.222.0.26 () {56 vars in 1376 bytes} [Wed Nov 2 03:21:07 2022] GET /identity/v3/role_assignments?group.id=a7530c4002db4ef4b7a83da573347759&scope.project.id=90ca3cc9c54d44feb93e6ed209b99483&role.id=addc0db6b4ac4362b07fe21e14f89180 => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:10.401215 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cb7440-7e57-4ea2-8d49-2f8bc4f753c9 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:10.401739 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cb7440-7e57-4ea2-8d49-2f8bc4f753c9 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:10.401739 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cb7440-7e57-4ea2-8d49-2f8bc4f753c9 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:10.403480 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 174/337] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:21:10.408752 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ecbfaf-376e-45dc-a17b-79f42091d96b None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:10.409165 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ecbfaf-376e-45dc-a17b-79f42091d96b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:10.409305 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ecbfaf-376e-45dc-a17b-79f42091d96b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:10.415848 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8a0f49-36a4-414d-80dc-79d770837592 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:10.416305 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8a0f49-36a4-414d-80dc-79d770837592 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:10.416649 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8a0f49-36a4-414d-80dc-79d770837592 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:10.418265 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 175/338] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 03:21:10.423254 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a47b23-f981-4aaa-a078-b914e8529467 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:10.423537 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a47b23-f981-4aaa-a078-b914e8529467 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:10.423816 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a47b23-f981-4aaa-a078-b914e8529467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:10.435800 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:10.447327 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-08ecbfaf-376e-45dc-a17b-79f42091d96b None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:10.448572 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-08ecbfaf-376e-45dc-a17b-79f42091d96b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:10.452389 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-69a47b23-f981-4aaa-a078-b914e8529467 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:10.453854 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-69a47b23-f981-4aaa-a078-b914e8529467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:10.538176 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:10.545142 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 164/339] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 03:21:10.546138 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 176/340] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Nov 02 03:21:10.546463 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf78a16f-c9af-48b8-9a6d-e2f38b66d21f None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:10.546748 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf78a16f-c9af-48b8-9a6d-e2f38b66d21f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:10.547017 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf78a16f-c9af-48b8-9a6d-e2f38b66d21f None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:10.556644 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 165/341] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:10 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:21:10.573613 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec1d1e-21c1-45e6-8260-89e2e4414bd1 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:10.574239 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec1d1e-21c1-45e6-8260-89e2e4414bd1 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:10.574717 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ec1d1e-21c1-45e6-8260-89e2e4414bd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:10.606498 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-91ec1d1e-21c1-45e6-8260-89e2e4414bd1 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:10.607693 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-91ec1d1e-21c1-45e6-8260-89e2e4414bd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:10.706049 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 177/342] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 03:21:10.837367 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf78a16f-c9af-48b8-9a6d-e2f38b66d21f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:10.915668 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ec1d1e-21c1-45e6-8260-89e2e4414bd1 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:11.002443 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf78a16f-c9af-48b8-9a6d-e2f38b66d21f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:11.005205 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:11.045261 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b961ee3b-59e1-43f4-9e61-19ef44511009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:11.046203 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961ee3b-59e1-43f4-9e61-19ef44511009 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:11.046654 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961ee3b-59e1-43f4-9e61-19ef44511009 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:11.047058 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ec1d1e-21c1-45e6-8260-89e2e4414bd1 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:11.047293 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961ee3b-59e1-43f4-9e61-19ef44511009 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:11.058256 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b961ee3b-59e1-43f4-9e61-19ef44511009 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:11.060161 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b961ee3b-59e1-43f4-9e61-19ef44511009 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:11.081466 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-b961ee3b-59e1-43f4-9e61-19ef44511009 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:11.081466 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:11.081466 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:11.081466 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:11.081466 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:11.081466 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:11.081466 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:11.081466 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:11.081466 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:11.081466 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:11.081466 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:11.081466 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:11.081466 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 03:21:11.081466 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 03:21:11.081466 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 03:21:11.081466 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 03:21:11.082484 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:11.082484 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:11.082484 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:11.082484 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:11.082484 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:11.082484 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:11.082484 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:11.082484 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:11.082484 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:11.082484 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:11.082484 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:11.082484 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:11.082484 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:11.082484 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:11.082484 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:11.082484 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:11.082484 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 03:21:11.083373 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 03:21:11.083373 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:11.083373 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:11.083671 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 166/343] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:21:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:11.085094 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1acfe-0f94-4b35-af52-7643910cd4b3 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:11.086337 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1acfe-0f94-4b35-af52-7643910cd4b3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:11.086337 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd1acfe-0f94-4b35-af52-7643910cd4b3 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:11.087488 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 167/344] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:21:11.088132 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd1acfe-0f94-4b35-af52-7643910cd4b3 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:11.089151 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479551b1-4f3c-4380-ac6c-561b22a674ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:11.089900 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479551b1-4f3c-4380-ac6c-561b22a674ce None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:11.090272 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479551b1-4f3c-4380-ac6c-561b22a674ce None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:11.090635 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479551b1-4f3c-4380-ac6c-561b22a674ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:11.101638 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479551b1-4f3c-4380-ac6c-561b22a674ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:11.103100 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479551b1-4f3c-4380-ac6c-561b22a674ce None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:11.117999 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-479551b1-4f3c-4380-ac6c-561b22a674ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:11.117999 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:11.117999 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:11.117999 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:11.117999 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:11.117999 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:11.117999 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:11.117999 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:11.117999 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:11.117999 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:11.117999 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:11.117999 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:11.117999 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 03:21:11.117999 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 03:21:11.117999 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 03:21:11.117999 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 03:21:11.118913 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:11.118913 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:11.118913 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:11.118913 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:11.118913 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:11.118913 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:11.118913 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:11.118913 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:11.118913 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:11.118913 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:11.118913 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:11.118913 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:11.118913 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:11.118913 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:11.118913 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:11.118913 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:11.118913 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 03:21:11.119801 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 03:21:11.119801 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:11.119801 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:11.119801 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 178/345] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:21:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:11.120061 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479551b1-4f3c-4380-ac6c-561b22a674ce None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:11.152078 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:11.206716 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd1acfe-0f94-4b35-af52-7643910cd4b3 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:11.225328 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479551b1-4f3c-4380-ac6c-561b22a674ce None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:11.242027 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:11.243807 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0afed7d-72f4-41fb-9bf6-6249e4349b83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:11.244591 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0afed7d-72f4-41fb-9bf6-6249e4349b83 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:11.244876 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0afed7d-72f4-41fb-9bf6-6249e4349b83 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:11.245165 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0afed7d-72f4-41fb-9bf6-6249e4349b83 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:11.256840 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0afed7d-72f4-41fb-9bf6-6249e4349b83 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:11.258755 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0afed7d-72f4-41fb-9bf6-6249e4349b83 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:11.262101 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a350c567-d926-4e24-8b82-a440eda5637f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:11.262708 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a350c567-d926-4e24-8b82-a440eda5637f None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:11.262987 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a350c567-d926-4e24-8b82-a440eda5637f None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:11.263256 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a350c567-d926-4e24-8b82-a440eda5637f None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:11.263899 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a350c567-d926-4e24-8b82-a440eda5637f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:11.264227 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a350c567-d926-4e24-8b82-a440eda5637f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:11.265561 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a350c567-d926-4e24-8b82-a440eda5637f None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:11.274069 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-d0afed7d-72f4-41fb-9bf6-6249e4349b83 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:11.274069 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:11.274069 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:11.274069 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:11.274069 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:11.274069 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:11.274069 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:11.274069 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:11.274069 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:11.274069 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:11.274069 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:11.274069 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:11.274069 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 03:21:11.274069 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 03:21:11.274069 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 03:21:11.274069 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 03:21:11.274951 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:11.274951 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:11.274951 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:11.274951 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:11.274951 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:11.274951 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:11.274951 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:11.274951 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:11.274951 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:11.274951 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:11.274951 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:11.274951 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:11.274951 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:11.274951 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:11.274951 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:11.274951 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:11.274951 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 03:21:11.275781 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 03:21:11.275781 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:11.275781 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:11.275781 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 168/346] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:21:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:11.279764 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 179/347] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Nov 2 03:21:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:11.280503 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a350c567-d926-4e24-8b82-a440eda5637f None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:11.285763 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf634623-09af-469f-9cc4-33e9aa5c7a05 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:11.285763 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf634623-09af-469f-9cc4-33e9aa5c7a05 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:11.285763 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf634623-09af-469f-9cc4-33e9aa5c7a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:11.306744 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-bf634623-09af-469f-9cc4-33e9aa5c7a05 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:11.307939 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-bf634623-09af-469f-9cc4-33e9aa5c7a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:11.388360 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 169/348] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 03:21:11.388360 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf634623-09af-469f-9cc4-33e9aa5c7a05 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:11.391043 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a350c567-d926-4e24-8b82-a440eda5637f None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:11.434883 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:11.438573 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77633cb1-a6ed-4bf5-b113-7bcda4110050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:11.439193 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77633cb1-a6ed-4bf5-b113-7bcda4110050 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:11.439474 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77633cb1-a6ed-4bf5-b113-7bcda4110050 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:11.439772 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77633cb1-a6ed-4bf5-b113-7bcda4110050 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:11.440584 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77633cb1-a6ed-4bf5-b113-7bcda4110050 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:11.440893 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77633cb1-a6ed-4bf5-b113-7bcda4110050 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:11.442276 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77633cb1-a6ed-4bf5-b113-7bcda4110050 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:11.460377 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 180/349] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Nov 2 03:21:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:11.461371 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77633cb1-a6ed-4bf5-b113-7bcda4110050 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:11.505686 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:11.527338 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf634623-09af-469f-9cc4-33e9aa5c7a05 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:11.570477 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a3ef93-4453-4db0-84ac-a100f4d9bd1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:11.571125 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3ef93-4453-4db0-84ac-a100f4d9bd1e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:11.571454 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3ef93-4453-4db0-84ac-a100f4d9bd1e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:11.571667 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a3ef93-4453-4db0-84ac-a100f4d9bd1e None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:11.572367 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a3ef93-4453-4db0-84ac-a100f4d9bd1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:11.572670 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a3ef93-4453-4db0-84ac-a100f4d9bd1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:11.574421 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a3ef93-4453-4db0-84ac-a100f4d9bd1e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:11.592607 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 170/350] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Nov 2 03:21:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:11.592607 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a3ef93-4453-4db0-84ac-a100f4d9bd1e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:11.602684 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77633cb1-a6ed-4bf5-b113-7bcda4110050 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:11.650540 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147ebf03-b281-4ef5-8ebc-44ccb3174c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:11.651690 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ebf03-b281-4ef5-8ebc-44ccb3174c40 None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:11.652148 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ebf03-b281-4ef5-8ebc-44ccb3174c40 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:11.653053 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ebf03-b281-4ef5-8ebc-44ccb3174c40 None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:11.654141 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147ebf03-b281-4ef5-8ebc-44ccb3174c40 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:11.656048 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147ebf03-b281-4ef5-8ebc-44ccb3174c40 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:11.685195 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 181/351] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Nov 2 03:21:11 2022] POST /identity/v3/users => generated 246 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:21:11.686004 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147ebf03-b281-4ef5-8ebc-44ccb3174c40 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:11.711962 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:11.720593 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a3ef93-4453-4db0-84ac-a100f4d9bd1e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:11.762122 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8c78b2-ec5f-4904-b666-b41fffaa34b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:11.762757 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8c78b2-ec5f-4904-b666-b41fffaa34b2 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:11.763038 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8c78b2-ec5f-4904-b666-b41fffaa34b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:11.763314 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8c78b2-ec5f-4904-b666-b41fffaa34b2 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:11.764010 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8c78b2-ec5f-4904-b666-b41fffaa34b2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:11.765523 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8c78b2-ec5f-4904-b666-b41fffaa34b2 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:11.796897 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 171/352] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Nov 2 03:21:11 2022] POST /identity/v3/users => generated 248 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:21:11.796897 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa907119-d71a-4b2b-9cf8-54d2edee60fc None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:11.796897 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa907119-d71a-4b2b-9cf8-54d2edee60fc None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:11.796897 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa907119-d71a-4b2b-9cf8-54d2edee60fc None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:11.796897 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 172/353] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:21:11.796897 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa907119-d71a-4b2b-9cf8-54d2edee60fc None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:11.802614 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:11.818889 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147ebf03-b281-4ef5-8ebc-44ccb3174c40 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:11.863942 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7db77e-3d4f-425b-b217-665889e034d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:11.864805 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7db77e-3d4f-425b-b217-665889e034d5 None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:11.865149 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7db77e-3d4f-425b-b217-665889e034d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:11.865493 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7db77e-3d4f-425b-b217-665889e034d5 None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:11.866241 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7db77e-3d4f-425b-b217-665889e034d5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:11.868038 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7db77e-3d4f-425b-b217-665889e034d5 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:11.896072 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 182/354] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Nov 2 03:21:11 2022] POST /identity/v3/users => generated 248 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:21:11.897470 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5807c43a-e814-461f-80a6-c439d101c3b7 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:11.897952 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5807c43a-e814-461f-80a6-c439d101c3b7 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:11.898294 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5807c43a-e814-461f-80a6-c439d101c3b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:11.924433 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-5807c43a-e814-461f-80a6-c439d101c3b7 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:11.926201 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-5807c43a-e814-461f-80a6-c439d101c3b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:11.950567 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa907119-d71a-4b2b-9cf8-54d2edee60fc None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:11.994956 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05b58ba-f831-43c5-bcdd-2922684be597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:11.995728 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b58ba-f831-43c5-bcdd-2922684be597 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:11.996134 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b58ba-f831-43c5-bcdd-2922684be597 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:11.996516 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05b58ba-f831-43c5-bcdd-2922684be597 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:12.011387 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05b58ba-f831-43c5-bcdd-2922684be597 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:12.013061 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05b58ba-f831-43c5-bcdd-2922684be597 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:12.019179 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 183/355] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 03:21:12.036494 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-d05b58ba-f831-43c5-bcdd-2922684be597 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:12.036494 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:12.036494 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:12.036494 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:12.036494 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:12.036494 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:12.036494 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:12.036494 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:12.036494 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:12.036494 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:12.036494 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:12.036494 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:12.036494 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 03:21:12.036494 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 03:21:12.036494 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 03:21:12.036494 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 03:21:12.037608 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:12.037608 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:12.037608 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:12.037608 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:12.037608 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:12.037608 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:12.037608 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:12.037608 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:12.037608 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:12.037608 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:12.037608 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:12.037608 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:12.037608 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:12.037608 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:12.037608 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:12.037608 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:12.037608 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 03:21:12.039299 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 03:21:12.039299 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:12.039299 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:12.039299 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 173/356] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:21:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:12.044207 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5807c43a-e814-461f-80a6-c439d101c3b7 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:12.135126 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:12.163628 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5807c43a-e814-461f-80a6-c439d101c3b7 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:12.201075 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dec44ad-216d-4919-896c-2a2db8dc60c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:12.201752 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dec44ad-216d-4919-896c-2a2db8dc60c3 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:12.202032 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dec44ad-216d-4919-896c-2a2db8dc60c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:12.202304 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dec44ad-216d-4919-896c-2a2db8dc60c3 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:12.202950 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dec44ad-216d-4919-896c-2a2db8dc60c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:12.203255 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dec44ad-216d-4919-896c-2a2db8dc60c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:12.204928 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dec44ad-216d-4919-896c-2a2db8dc60c3 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:12.218103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 184/357] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Nov 2 03:21:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:12.223678 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05b58ba-f831-43c5-bcdd-2922684be597 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:12.226904 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:12.324614 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dec44ad-216d-4919-896c-2a2db8dc60c3 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:12.340306 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05b58ba-f831-43c5-bcdd-2922684be597 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:12.376022 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebbf7d4-f55b-4003-b4b7-e6c5b5462bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:12.376999 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebbf7d4-f55b-4003-b4b7-e6c5b5462bd1 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:12.377478 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebbf7d4-f55b-4003-b4b7-e6c5b5462bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:12.377962 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebbf7d4-f55b-4003-b4b7-e6c5b5462bd1 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:12.378825 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebbf7d4-f55b-4003-b4b7-e6c5b5462bd1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:12.380471 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebbf7d4-f55b-4003-b4b7-e6c5b5462bd1 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:12.404655 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 174/358] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Nov 2 03:21:12 2022] POST /identity/v3/users => generated 249 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:21:12.479215 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dec44ad-216d-4919-896c-2a2db8dc60c3 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:12.538121 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8512d2-60c8-4d31-aa9c-20a660190c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:12.539262 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8512d2-60c8-4d31-aa9c-20a660190c49 None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:12.539543 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8512d2-60c8-4d31-aa9c-20a660190c49 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:12.539818 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8512d2-60c8-4d31-aa9c-20a660190c49 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:12.540543 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8512d2-60c8-4d31-aa9c-20a660190c49 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:12.541929 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8512d2-60c8-4d31-aa9c-20a660190c49 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:12.555755 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.sql.core [None req-3d8512d2-60c8-4d31-aa9c-20a660190c49 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Nov 02 03:21:12.555755 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Nov 02 03:21:12.555755 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [parameters: {'id': 'b9d20ad4565d40f590a50d7ef33bb0d2', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Nov 02 03:21:12.555755 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81504) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Nov 02 03:21:12.572575 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-3d8512d2-60c8-4d31-aa9c-20a660190c49 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Nov 02 03:21:12.572575 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:12.572575 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 03:21:12.572575 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 03:21:12.572575 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 03:21:12.572575 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 03:21:12.572575 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 03:21:12.572575 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 03:21:12.572575 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 03:21:12.572575 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application conn.query(q) Nov 02 03:21:12.572575 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 03:21:12.572575 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 03:21:12.572575 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 03:21:12.572575 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application result.read() Nov 02 03:21:12.572575 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 03:21:12.573514 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 03:21:12.573514 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 03:21:12.573514 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 03:21:12.573514 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 03:21:12.573514 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 03:21:12.573514 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 03:21:12.573514 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 03:21:12.573514 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Nov 02 03:21:12.573514 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:12.573514 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Nov 02 03:21:12.573514 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:12.573514 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:12.573514 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Nov 02 03:21:12.573514 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return method(*args, **kwargs) Nov 02 03:21:12.573514 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Nov 02 03:21:12.573514 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return ref.to_dict() Nov 02 03:21:12.573514 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 03:21:12.573514 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application next(self.gen) Nov 02 03:21:12.574427 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Nov 02 03:21:12.574427 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application yield resource Nov 02 03:21:12.574427 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 03:21:12.574427 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application next(self.gen) Nov 02 03:21:12.574427 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Nov 02 03:21:12.574427 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self.session.rollback() Nov 02 03:21:12.574427 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Nov 02 03:21:12.574427 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self.force_reraise() Nov 02 03:21:12.574427 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Nov 02 03:21:12.574427 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise self.value Nov 02 03:21:12.574427 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Nov 02 03:21:12.574427 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Nov 02 03:21:12.574427 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Nov 02 03:21:12.574427 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application session.commit() Nov 02 03:21:12.574427 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Nov 02 03:21:12.574427 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Nov 02 03:21:12.574427 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Nov 02 03:21:12.575914 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._prepare_impl() Nov 02 03:21:12.575914 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Nov 02 03:21:12.575914 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self.session.flush() Nov 02 03:21:12.575914 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Nov 02 03:21:12.575914 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._flush(objects) Nov 02 03:21:12.575914 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Nov 02 03:21:12.575914 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Nov 02 03:21:12.575914 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Nov 02 03:21:12.575914 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application compat.raise_( Nov 02 03:21:12.575914 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 03:21:12.575914 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception Nov 02 03:21:12.575914 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Nov 02 03:21:12.575914 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application flush_context.execute() Nov 02 03:21:12.575914 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Nov 02 03:21:12.575914 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rec.execute(self) Nov 02 03:21:12.575914 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Nov 02 03:21:12.575914 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Nov 02 03:21:12.577398 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Nov 02 03:21:12.577398 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application _emit_insert_statements( Nov 02 03:21:12.577398 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Nov 02 03:21:12.577398 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application c = connection._execute_20( Nov 02 03:21:12.577398 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Nov 02 03:21:12.577398 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Nov 02 03:21:12.577398 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Nov 02 03:21:12.577398 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Nov 02 03:21:12.577398 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Nov 02 03:21:12.577398 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ret = self._execute_context( Nov 02 03:21:12.577398 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Nov 02 03:21:12.577398 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Nov 02 03:21:12.577398 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Nov 02 03:21:12.577398 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Nov 02 03:21:12.577398 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 03:21:12.577398 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception Nov 02 03:21:12.579278 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 03:21:12.579278 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 03:21:12.579278 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 03:21:12.579278 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 03:21:12.579278 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 03:21:12.579278 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 03:21:12.579278 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 03:21:12.579278 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application conn.query(q) Nov 02 03:21:12.579278 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 03:21:12.579278 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 03:21:12.579278 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 03:21:12.579278 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application result.read() Nov 02 03:21:12.579278 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 03:21:12.579278 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 03:21:12.579278 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 03:21:12.579278 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 03:21:12.581244 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 03:21:12.581244 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 03:21:12.581244 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 03:21:12.581244 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 03:21:12.581244 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Nov 02 03:21:12.581244 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Nov 02 03:21:12.581244 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [parameters: {'id': 'b9d20ad4565d40f590a50d7ef33bb0d2', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Nov 02 03:21:12.581244 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Nov 02 03:21:12.581244 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:12.581244 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 03:21:12.581244 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:12.581244 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:12.581244 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:12.581244 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:12.581244 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:12.581244 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:12.583236 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:12.583236 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:12.583236 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:12.583236 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:12.583236 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:12.583236 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:12.583236 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Nov 02 03:21:12.583236 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Nov 02 03:21:12.583236 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:12.583236 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:12.583236 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Nov 02 03:21:12.583236 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Nov 02 03:21:12.583236 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Nov 02 03:21:12.583236 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Nov 02 03:21:12.583236 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Nov 02 03:21:12.583236 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:12.585175 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 185/359] 10.222.0.26 () {60 vars in 1131 bytes} [Wed Nov 2 03:21:12 2022] POST /identity/v3/roles => generated 144 bytes in 250 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Nov 02 03:21:12.585382 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebbf7d4-f55b-4003-b4b7-e6c5b5462bd1 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:12.646136 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:12.715195 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebbf7d4-f55b-4003-b4b7-e6c5b5462bd1 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:12.755358 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19fefb1-0e24-4f6a-9fcd-61de4c6eeb32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:12.756079 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19fefb1-0e24-4f6a-9fcd-61de4c6eeb32 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:12.756357 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19fefb1-0e24-4f6a-9fcd-61de4c6eeb32 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:12.756628 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19fefb1-0e24-4f6a-9fcd-61de4c6eeb32 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:12.771819 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19fefb1-0e24-4f6a-9fcd-61de4c6eeb32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:21:12.777503 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-b19fefb1-0e24-4f6a-9fcd-61de4c6eeb32 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 03:21:12.777503 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:12.777503 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:12.777503 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:12.777503 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:12.777503 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:12.777503 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:12.777503 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:12.777503 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:12.777503 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:12.777503 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:12.777503 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:12.777503 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:21:12.777503 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:21:12.777503 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:21:12.777503 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:21:12.778396 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:21:12.778396 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:21:12.778396 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:21:12.778396 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:21:12.778396 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:12.778396 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:12.778396 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:12.778396 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:12.778396 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:12.778396 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:12.778396 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:12.778396 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:12.778396 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:12.778396 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:12.778396 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:12.778396 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:12.779197 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:12.779197 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:12.779197 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:12.779197 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:12.779197 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:21:12.779197 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:21:12.779197 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:21:12.779197 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:21:12.779197 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:21:12.779197 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:21:12.779197 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Nov 02 03:21:12.779197 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:12.779197 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 175/360] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Nov 2 03:21:12 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:12.786105 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8512d2-60c8-4d31-aa9c-20a660190c49 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:12.791941 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:12.900835 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8512d2-60c8-4d31-aa9c-20a660190c49 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:12.941798 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5c9e84-4352-4518-a1c2-8e3a7213502a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:12.942801 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5c9e84-4352-4518-a1c2-8e3a7213502a None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:12.943279 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5c9e84-4352-4518-a1c2-8e3a7213502a None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:12.943748 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5c9e84-4352-4518-a1c2-8e3a7213502a None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:12.944714 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5c9e84-4352-4518-a1c2-8e3a7213502a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:12.945733 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5c9e84-4352-4518-a1c2-8e3a7213502a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:12.947472 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5c9e84-4352-4518-a1c2-8e3a7213502a None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:12.959366 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 186/361] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Nov 2 03:21:12 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:14.319284 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be7f0d-9ef6-405c-835d-0540f367f1b3 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:14.320345 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be7f0d-9ef6-405c-835d-0540f367f1b3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:14.320451 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be7f0d-9ef6-405c-835d-0540f367f1b3 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:14.321982 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 176/362] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:21:14.327051 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d233674a-26eb-4105-8b6f-a350a79d9f01 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:14.327467 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d233674a-26eb-4105-8b6f-a350a79d9f01 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:14.327634 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d233674a-26eb-4105-8b6f-a350a79d9f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:14.346245 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-d233674a-26eb-4105-8b6f-a350a79d9f01 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:14.347430 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d233674a-26eb-4105-8b6f-a350a79d9f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:14.429395 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:14.443641 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 187/363] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 03:21:14.550979 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f61212-22e3-4470-bea3-df771a9c71ef None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:14.551260 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f61212-22e3-4470-bea3-df771a9c71ef None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:14.551539 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f61212-22e3-4470-bea3-df771a9c71ef None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:14.553266 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 177/364] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:21:14.559130 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757d6a9-47a9-41f5-bfd3-503f1c3f4eb2 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:14.559688 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757d6a9-47a9-41f5-bfd3-503f1c3f4eb2 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:14.560206 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b757d6a9-47a9-41f5-bfd3-503f1c3f4eb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:14.584308 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-b757d6a9-47a9-41f5-bfd3-503f1c3f4eb2 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:14.586297 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-b757d6a9-47a9-41f5-bfd3-503f1c3f4eb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:14.656223 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a22eff-43ca-4bb9-bf9e-d763724184c2 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:14.656560 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a22eff-43ca-4bb9-bf9e-d763724184c2 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:14.656796 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a22eff-43ca-4bb9-bf9e-d763724184c2 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:14.658118 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 178/365] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:21:14.663969 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12adedc3-a834-47ed-ab52-3825dcbd59a8 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:14.664263 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12adedc3-a834-47ed-ab52-3825dcbd59a8 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:14.664535 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12adedc3-a834-47ed-ab52-3825dcbd59a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:14.693488 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-12adedc3-a834-47ed-ab52-3825dcbd59a8 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:14.695600 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-12adedc3-a834-47ed-ab52-3825dcbd59a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:14.701079 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 188/366] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 03:21:14.768164 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b757d6a9-47a9-41f5-bfd3-503f1c3f4eb2 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:14.793202 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 179/367] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 03:21:14.922558 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b757d6a9-47a9-41f5-bfd3-503f1c3f4eb2 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:14.955518 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:14.967460 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b7136a-05b5-4782-a00e-3bbe50d1b9ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:14.968136 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b7136a-05b5-4782-a00e-3bbe50d1b9ee None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:14.968415 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b7136a-05b5-4782-a00e-3bbe50d1b9ee None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:14.968684 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b7136a-05b5-4782-a00e-3bbe50d1b9ee None admin] PATH_INFO: `/v3/roles/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:14.980287 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12adedc3-a834-47ed-ab52-3825dcbd59a8 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:14.988115 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b7136a-05b5-4782-a00e-3bbe50d1b9ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:21:14.997869 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-f7b7136a-05b5-4782-a00e-3bbe50d1b9ee None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 03:21:14.997869 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:14.997869 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:14.997869 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:14.997869 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:14.997869 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:14.997869 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:14.997869 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:14.997869 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:14.997869 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:14.997869 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:14.997869 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:14.997869 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:21:14.997869 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:21:14.997869 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:21:14.997869 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:21:14.998753 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:21:14.998753 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:21:14.998753 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:21:14.998753 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:21:14.998753 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:14.998753 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:14.998753 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:14.998753 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:14.998753 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:14.998753 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:14.998753 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:14.998753 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:14.998753 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:14.998753 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:14.998753 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:14.998753 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:14.999581 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:14.999581 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:14.999581 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:14.999581 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:14.999581 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:21:14.999581 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:21:14.999581 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:21:14.999581 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:21:14.999581 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:21:14.999581 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:21:14.999581 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 03:21:14.999581 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:15.006306 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 189/368] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Nov 2 03:21:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 03:21:15.010903 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b7136a-05b5-4782-a00e-3bbe50d1b9ee None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:15.063732 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:15.154046 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12adedc3-a834-47ed-ab52-3825dcbd59a8 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:15.183041 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b7136a-05b5-4782-a00e-3bbe50d1b9ee None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:15.200139 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d94d71-6766-4093-a95f-0732b8481f1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:15.201519 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d94d71-6766-4093-a95f-0732b8481f1a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:15.202268 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d94d71-6766-4093-a95f-0732b8481f1a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:15.202751 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d94d71-6766-4093-a95f-0732b8481f1a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:15.218402 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d94d71-6766-4093-a95f-0732b8481f1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:21:15.225842 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-60d94d71-6766-4093-a95f-0732b8481f1a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 03:21:15.225842 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:15.225842 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:15.225842 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:15.225842 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:15.225842 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:15.225842 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:15.225842 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:15.225842 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:15.225842 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:15.225842 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:15.225842 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:15.225842 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:21:15.225842 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:21:15.225842 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:21:15.225842 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:21:15.226876 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:21:15.226876 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:21:15.226876 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:21:15.226876 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:21:15.226876 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:15.226876 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:15.226876 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:15.226876 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:15.226876 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:15.226876 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:15.226876 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:15.226876 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:15.226876 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:15.226876 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:15.226876 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:15.226876 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:15.227778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:15.227778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:15.227778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:15.227778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:15.227778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:21:15.227778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:21:15.227778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:21:15.227778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:21:15.227778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:21:15.227778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:21:15.227778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 03:21:15.227778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:15.229485 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 180/369] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Nov 2 03:21:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:15.231323 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d94d71-6766-4093-a95f-0732b8481f1a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:15.232395 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b9a352-0c72-438a-ab22-06d279612b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:15.233006 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b9a352-0c72-438a-ab22-06d279612b89 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:15.233285 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b9a352-0c72-438a-ab22-06d279612b89 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:15.233578 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b9a352-0c72-438a-ab22-06d279612b89 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:15.234691 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b9a352-0c72-438a-ab22-06d279612b89 None admin] RBAC: Adding query filter params (name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:15.234991 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b9a352-0c72-438a-ab22-06d279612b89 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:15.236388 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b9a352-0c72-438a-ab22-06d279612b89 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:15.248652 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 190/370] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Nov 2 03:21:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:15.249565 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b9a352-0c72-438a-ab22-06d279612b89 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:15.314850 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:15.352990 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d94d71-6766-4093-a95f-0732b8481f1a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:15.362473 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:15.380417 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b9a352-0c72-438a-ab22-06d279612b89 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:15.388905 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22d00f4-9efd-480b-b3b4-5dfdcb19b229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:15.389553 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d00f4-9efd-480b-b3b4-5dfdcb19b229 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:15.389826 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d00f4-9efd-480b-b3b4-5dfdcb19b229 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:15.390094 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d00f4-9efd-480b-b3b4-5dfdcb19b229 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:15.401136 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22d00f4-9efd-480b-b3b4-5dfdcb19b229 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:21:15.406860 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-f22d00f4-9efd-480b-b3b4-5dfdcb19b229 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 03:21:15.406860 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:15.406860 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:15.406860 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:15.406860 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:15.406860 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:15.406860 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:15.406860 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:15.406860 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:15.406860 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:15.406860 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:15.406860 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:15.406860 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:21:15.406860 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:21:15.406860 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:21:15.406860 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:21:15.407778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:21:15.407778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:21:15.407778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:21:15.407778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:21:15.407778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:15.407778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:15.407778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:15.407778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:15.407778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:15.407778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:15.407778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:15.407778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:15.407778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:15.407778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:15.407778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:15.407778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:15.408633 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:15.408633 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:15.408633 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:15.408633 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:15.408633 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:21:15.408633 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:21:15.408633 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:21:15.408633 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:21:15.408633 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:21:15.408633 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:21:15.408633 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 03:21:15.408633 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:15.408633 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 181/371] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Nov 2 03:21:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:15.409350 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22d00f4-9efd-480b-b3b4-5dfdcb19b229 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:15.426921 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2670a31-6b03-462e-80ad-d2df66614cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:15.427569 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2670a31-6b03-462e-80ad-d2df66614cc0 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:15.427826 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2670a31-6b03-462e-80ad-d2df66614cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:15.428146 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2670a31-6b03-462e-80ad-d2df66614cc0 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:15.428820 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2670a31-6b03-462e-80ad-d2df66614cc0 None admin] RBAC: Adding query filter params (name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:15.429695 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2670a31-6b03-462e-80ad-d2df66614cc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:15.431138 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2670a31-6b03-462e-80ad-d2df66614cc0 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:15.439838 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 191/372] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Nov 2 03:21:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:15.440873 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e157f-acd4-4be7-8cda-74f5fe32c7ce None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:15.441162 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e157f-acd4-4be7-8cda-74f5fe32c7ce None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:15.441441 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e157f-acd4-4be7-8cda-74f5fe32c7ce None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:15.442640 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 192/373] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:21:15.445562 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69e157f-acd4-4be7-8cda-74f5fe32c7ce None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:15.515143 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22d00f4-9efd-480b-b3b4-5dfdcb19b229 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:15.555339 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69e157f-acd4-4be7-8cda-74f5fe32c7ce None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:15.556886 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b90605-efbf-4453-a36d-7ebe2115aaca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:15.557415 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b90605-efbf-4453-a36d-7ebe2115aaca None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:15.557695 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b90605-efbf-4453-a36d-7ebe2115aaca None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:15.557973 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b90605-efbf-4453-a36d-7ebe2115aaca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:15.565572 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:15.569079 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b90605-efbf-4453-a36d-7ebe2115aaca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:15.571130 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b90605-efbf-4453-a36d-7ebe2115aaca None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:15.586380 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-f1b90605-efbf-4453-a36d-7ebe2115aaca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:15.586380 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:15.586380 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:15.586380 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:15.586380 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:15.586380 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:15.586380 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:15.586380 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:15.586380 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:15.586380 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:15.586380 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:15.586380 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:15.586380 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 03:21:15.586380 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 03:21:15.586380 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 03:21:15.586380 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 03:21:15.587328 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:15.587328 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:15.587328 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:15.587328 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:15.587328 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:15.587328 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:15.587328 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:15.587328 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:15.587328 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:15.587328 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:15.587328 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:15.587328 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:15.587328 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:15.587328 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:15.587328 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:15.587328 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:15.587328 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 03:21:15.588302 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 03:21:15.588302 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:15.588302 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:15.588302 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 182/374] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:21:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:15.592727 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b90605-efbf-4453-a36d-7ebe2115aaca None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:15.594521 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e5f3fe-3661-4549-be81-76b42a4b28bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:15.595146 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e5f3fe-3661-4549-be81-76b42a4b28bd None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:15.595448 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e5f3fe-3661-4549-be81-76b42a4b28bd None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:15.595691 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e5f3fe-3661-4549-be81-76b42a4b28bd None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:15.596962 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e5f3fe-3661-4549-be81-76b42a4b28bd None admin] RBAC: Adding query filter params (name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:15.596962 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e5f3fe-3661-4549-be81-76b42a4b28bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:15.598045 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e5f3fe-3661-4549-be81-76b42a4b28bd None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:15.607550 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 193/375] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Nov 2 03:21:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:15.609971 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c580d771-ac88-44bf-bbfb-0a9f5613b366 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:15.610458 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c580d771-ac88-44bf-bbfb-0a9f5613b366 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:15.611183 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c580d771-ac88-44bf-bbfb-0a9f5613b366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:15.616666 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:15.629973 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-c580d771-ac88-44bf-bbfb-0a9f5613b366 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:15.631160 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-c580d771-ac88-44bf-bbfb-0a9f5613b366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:15.700026 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b90605-efbf-4453-a36d-7ebe2115aaca None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:15.703260 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 194/376] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 03:21:15.704887 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c580d771-ac88-44bf-bbfb-0a9f5613b366 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:15.735577 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40b5eda-09f1-4087-8157-9e902406952a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:15.736267 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b5eda-09f1-4087-8157-9e902406952a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:15.736606 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b5eda-09f1-4087-8157-9e902406952a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:15.736812 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40b5eda-09f1-4087-8157-9e902406952a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:15.749337 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40b5eda-09f1-4087-8157-9e902406952a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:15.750839 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40b5eda-09f1-4087-8157-9e902406952a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:15.767771 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-f40b5eda-09f1-4087-8157-9e902406952a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:15.767771 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:15.767771 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:15.767771 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:15.767771 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:15.767771 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:15.767771 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:15.767771 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:15.767771 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:15.767771 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:15.767771 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:15.767771 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:15.767771 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 03:21:15.767771 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 03:21:15.767771 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 03:21:15.767771 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 03:21:15.768894 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:15.768894 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:15.768894 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:15.768894 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:15.768894 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:15.768894 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:15.768894 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:15.768894 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:15.768894 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:15.768894 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:15.768894 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:15.768894 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:15.768894 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:15.768894 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:15.768894 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:15.768894 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:15.768894 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 03:21:15.769738 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 03:21:15.769738 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:15.769738 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:15.769738 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 183/377] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:21:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:15.769974 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40b5eda-09f1-4087-8157-9e902406952a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:15.836310 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:15.839463 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c580d771-ac88-44bf-bbfb-0a9f5613b366 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:15.867499 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:15.876886 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941e6ec2-8934-48ea-ae5f-f9a153afb2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:15.877506 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e6ec2-8934-48ea-ae5f-f9a153afb2ee None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:15.877784 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e6ec2-8934-48ea-ae5f-f9a153afb2ee None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:15.878056 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941e6ec2-8934-48ea-ae5f-f9a153afb2ee None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:15.878700 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941e6ec2-8934-48ea-ae5f-f9a153afb2ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:15.879006 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941e6ec2-8934-48ea-ae5f-f9a153afb2ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:15.880425 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941e6ec2-8934-48ea-ae5f-f9a153afb2ee None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:15.895940 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 195/378] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Nov 2 03:21:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:15.895940 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941e6ec2-8934-48ea-ae5f-f9a153afb2ee None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:15.903799 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40b5eda-09f1-4087-8157-9e902406952a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:15.943888 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f911d35b-0b72-41e3-b237-78ff248199bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:15.944561 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f911d35b-0b72-41e3-b237-78ff248199bb None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:15.945377 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f911d35b-0b72-41e3-b237-78ff248199bb None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:15.945653 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f911d35b-0b72-41e3-b237-78ff248199bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:15.957221 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f911d35b-0b72-41e3-b237-78ff248199bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:15.958588 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f911d35b-0b72-41e3-b237-78ff248199bb None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:15.973867 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-f911d35b-0b72-41e3-b237-78ff248199bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:15.973867 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:15.973867 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:15.973867 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:15.973867 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:15.973867 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:15.973867 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:15.973867 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:15.973867 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:15.973867 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:15.973867 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:15.973867 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:15.973867 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 03:21:15.973867 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 03:21:15.973867 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 03:21:15.973867 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 03:21:15.974778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:15.974778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:15.974778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:15.974778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:15.974778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:15.974778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:15.974778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:15.974778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:15.974778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:15.974778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:15.974778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:15.974778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:15.974778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:15.974778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:15.974778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:15.974778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:15.974778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 03:21:15.975627 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 03:21:15.975627 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:15.975627 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:15.975627 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 184/379] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:21:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:15.976202 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c34f2-edbe-44b4-afb4-9f4800f703b6 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:15.976419 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c34f2-edbe-44b4-afb4-9f4800f703b6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:15.976691 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7c34f2-edbe-44b4-afb4-9f4800f703b6 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:15.977946 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 185/380] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:21:15.978631 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7c34f2-edbe-44b4-afb4-9f4800f703b6 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:16.002723 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941e6ec2-8934-48ea-ae5f-f9a153afb2ee None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:16.035967 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc2ee8d-896b-425f-8f0b-3e85b2f45d14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:16.036582 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc2ee8d-896b-425f-8f0b-3e85b2f45d14 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:16.036858 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc2ee8d-896b-425f-8f0b-3e85b2f45d14 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:16.037128 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc2ee8d-896b-425f-8f0b-3e85b2f45d14 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:16.037769 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc2ee8d-896b-425f-8f0b-3e85b2f45d14 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:16.038083 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc2ee8d-896b-425f-8f0b-3e85b2f45d14 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:16.039474 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc2ee8d-896b-425f-8f0b-3e85b2f45d14 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:16.050835 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 196/381] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Nov 2 03:21:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:16.051430 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc2ee8d-896b-425f-8f0b-3e85b2f45d14 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:16.073620 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:16.080495 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7c34f2-edbe-44b4-afb4-9f4800f703b6 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:16.104054 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:16.115410 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7103bda2-52ed-47f7-a266-346b1c385ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:16.116056 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7103bda2-52ed-47f7-a266-346b1c385ec4 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:16.116333 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7103bda2-52ed-47f7-a266-346b1c385ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:16.117174 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7103bda2-52ed-47f7-a266-346b1c385ec4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:16.124322 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7103bda2-52ed-47f7-a266-346b1c385ec4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:16.125743 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7103bda2-52ed-47f7-a266-346b1c385ec4 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:16.137761 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-7103bda2-52ed-47f7-a266-346b1c385ec4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Nov 02 03:21:16.137761 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:16.137761 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:16.137761 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:16.137761 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:16.137761 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:16.137761 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:16.137761 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:16.137761 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:16.137761 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:16.137761 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:16.137761 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:16.137761 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 03:21:16.137761 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 03:21:16.137761 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 03:21:16.137761 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 03:21:16.138664 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:16.138664 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:16.138664 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 03:21:16.138664 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 03:21:16.138664 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 03:21:16.138664 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 03:21:16.138664 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:16.138664 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:16.138664 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:16.138664 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:16.138664 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:16.138664 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:16.138664 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:16.138664 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:16.138664 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:16.138664 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:16.138664 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:16.138664 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:16.139621 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:16.139621 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:16.139621 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 03:21:16.139621 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 03:21:16.139621 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 03:21:16.139621 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 03:21:16.139621 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 03:21:16.139621 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 03:21:16.139621 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Nov 02 03:21:16.139621 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:16.139621 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 186/382] 10.222.0.26 () {56 vars in 1093 bytes} [Wed Nov 2 03:21:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:16.140240 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7103bda2-52ed-47f7-a266-346b1c385ec4 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:16.153276 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc2ee8d-896b-425f-8f0b-3e85b2f45d14 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:16.186480 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38f8564-f9ec-402c-b7af-cafb1941cc93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:16.187101 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f8564-f9ec-402c-b7af-cafb1941cc93 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:16.187375 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f8564-f9ec-402c-b7af-cafb1941cc93 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:16.187641 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f8564-f9ec-402c-b7af-cafb1941cc93 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:16.197568 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38f8564-f9ec-402c-b7af-cafb1941cc93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:21:16.202427 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-b38f8564-f9ec-402c-b7af-cafb1941cc93 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 03:21:16.202427 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:16.202427 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:16.202427 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:16.202427 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:16.202427 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:16.202427 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:16.202427 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:16.202427 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:16.202427 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:16.202427 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:16.202427 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:16.202427 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:21:16.202427 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:21:16.202427 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:21:16.202427 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:21:16.203474 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:21:16.203474 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:21:16.203474 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:21:16.203474 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:21:16.203474 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:16.203474 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:16.203474 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:16.203474 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:16.203474 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:16.203474 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:16.203474 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:16.203474 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:16.203474 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:16.203474 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:16.203474 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:16.203474 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:16.204509 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:16.204509 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:16.204509 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:16.204509 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:16.204509 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:21:16.204509 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:21:16.204509 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:21:16.204509 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:21:16.204509 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:21:16.204509 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:21:16.204509 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 03:21:16.204509 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:16.204509 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 197/383] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Nov 2 03:21:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:16.205335 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117beb0c-eff6-4649-8fe0-106a3bf84edc None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:16.205335 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117beb0c-eff6-4649-8fe0-106a3bf84edc None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:16.205335 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117beb0c-eff6-4649-8fe0-106a3bf84edc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:16.223873 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-117beb0c-eff6-4649-8fe0-106a3bf84edc None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:16.225117 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-117beb0c-eff6-4649-8fe0-106a3bf84edc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:16.245758 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7103bda2-52ed-47f7-a266-346b1c385ec4 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:16.278274 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d10978-4e4b-4e09-8905-a0f34df64d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:16.278876 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d10978-4e4b-4e09-8905-a0f34df64d0c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:16.279153 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d10978-4e4b-4e09-8905-a0f34df64d0c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:16.279424 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d10978-4e4b-4e09-8905-a0f34df64d0c None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:16.280087 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d10978-4e4b-4e09-8905-a0f34df64d0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:16.280389 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d10978-4e4b-4e09-8905-a0f34df64d0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:16.281761 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d10978-4e4b-4e09-8905-a0f34df64d0c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:16.290870 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 198/384] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 03:21:16.291690 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117beb0c-eff6-4649-8fe0-106a3bf84edc None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:16.298192 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 187/385] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Nov 2 03:21:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:16.298192 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d10978-4e4b-4e09-8905-a0f34df64d0c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:16.308828 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:16.349937 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:16.414786 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117beb0c-eff6-4649-8fe0-106a3bf84edc None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:16.420270 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d10978-4e4b-4e09-8905-a0f34df64d0c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:16.452663 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0565c578-e68a-4d26-9211-ed3b3be39b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:16.453302 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0565c578-e68a-4d26-9211-ed3b3be39b24 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:16.453580 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0565c578-e68a-4d26-9211-ed3b3be39b24 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:16.453848 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0565c578-e68a-4d26-9211-ed3b3be39b24 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:16.457802 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1090ed08-911f-468e-b28a-bd5e091b8c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:16.458603 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1090ed08-911f-468e-b28a-bd5e091b8c09 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:16.459056 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1090ed08-911f-468e-b28a-bd5e091b8c09 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:16.459527 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1090ed08-911f-468e-b28a-bd5e091b8c09 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:16.460449 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1090ed08-911f-468e-b28a-bd5e091b8c09 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:16.460922 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1090ed08-911f-468e-b28a-bd5e091b8c09 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:16.461461 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0565c578-e68a-4d26-9211-ed3b3be39b24 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:16.462812 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1090ed08-911f-468e-b28a-bd5e091b8c09 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:16.462935 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0565c578-e68a-4d26-9211-ed3b3be39b24 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:16.475287 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-0565c578-e68a-4d26-9211-ed3b3be39b24 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Nov 02 03:21:16.475287 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:16.475287 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:16.475287 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:16.475287 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:16.475287 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:16.475287 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:16.475287 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:16.475287 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:16.475287 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:16.475287 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:16.475287 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:16.475287 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 03:21:16.475287 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 03:21:16.475287 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 03:21:16.475287 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 03:21:16.476804 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:16.476804 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:16.476804 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 03:21:16.476804 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 03:21:16.476804 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 03:21:16.476804 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 03:21:16.476804 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:16.476804 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:16.476804 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:16.476804 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:16.476804 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:16.476804 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:16.476804 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:16.476804 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:16.476804 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:16.476804 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:16.476804 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:16.476804 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:16.477710 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:16.477710 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:16.477710 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 03:21:16.477710 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 03:21:16.477710 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 03:21:16.477710 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 03:21:16.477710 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 03:21:16.477710 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 03:21:16.477710 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Nov 02 03:21:16.477710 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:16.477710 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 199/386] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Nov 2 03:21:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:16.478276 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0565c578-e68a-4d26-9211-ed3b3be39b24 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:16.505408 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 188/387] 10.222.0.26 () {56 vars in 1133 bytes} [Wed Nov 2 03:21:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:16.506248 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1090ed08-911f-468e-b28a-bd5e091b8c09 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:16.568967 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:16.605538 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0565c578-e68a-4d26-9211-ed3b3be39b24 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:16.619027 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1090ed08-911f-468e-b28a-bd5e091b8c09 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:16.620615 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:16.646718 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bae4a4-d1c6-456c-8204-0105932d4b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:16.648175 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bae4a4-d1c6-456c-8204-0105932d4b9c None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:16.648491 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bae4a4-d1c6-456c-8204-0105932d4b9c None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:16.648768 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bae4a4-d1c6-456c-8204-0105932d4b9c None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:16.650332 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bae4a4-d1c6-456c-8204-0105932d4b9c None admin] RBAC: Adding query filter params (name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:16.650663 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bae4a4-d1c6-456c-8204-0105932d4b9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:16.652083 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bae4a4-d1c6-456c-8204-0105932d4b9c None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:16.656822 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1409db-20b6-4d8e-bcef-51ae74d9c79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:16.657458 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1409db-20b6-4d8e-bcef-51ae74d9c79b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:16.657733 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1409db-20b6-4d8e-bcef-51ae74d9c79b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:16.658001 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac1409db-20b6-4d8e-bcef-51ae74d9c79b None admin] PATH_INFO: `/v3/users/glance` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:16.664816 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 200/388] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Nov 2 03:21:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:16.665642 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bae4a4-d1c6-456c-8204-0105932d4b9c None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:16.666055 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1409db-20b6-4d8e-bcef-51ae74d9c79b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:16.667514 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac1409db-20b6-4d8e-bcef-51ae74d9c79b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:16.682806 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-ac1409db-20b6-4d8e-bcef-51ae74d9c79b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Nov 02 03:21:16.682806 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:16.682806 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:16.682806 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:16.682806 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:16.682806 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:16.682806 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:16.682806 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:16.682806 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:16.682806 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:16.682806 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:16.682806 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:16.682806 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 03:21:16.682806 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 03:21:16.682806 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 03:21:16.682806 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 03:21:16.683807 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:16.683807 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:16.683807 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 03:21:16.683807 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 03:21:16.683807 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 03:21:16.683807 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 03:21:16.683807 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:16.683807 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:16.683807 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:16.683807 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:16.683807 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:16.683807 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:16.683807 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:16.683807 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:16.683807 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:16.683807 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:16.683807 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:16.683807 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:16.684749 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:16.684749 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:16.684749 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 03:21:16.684749 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 03:21:16.684749 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 03:21:16.684749 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 03:21:16.684749 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 03:21:16.684749 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 03:21:16.684749 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Nov 02 03:21:16.684749 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:16.684749 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 189/389] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Nov 2 03:21:16 2022] GET /identity/v3/users/glance => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:16.684749 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1409db-20b6-4d8e-bcef-51ae74d9c79b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:16.767703 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bae4a4-d1c6-456c-8204-0105932d4b9c None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:16.793957 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac1409db-20b6-4d8e-bcef-51ae74d9c79b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:16.797563 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:16.802850 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78519647-bfc6-4f2f-a16f-b2109a5b31f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:16.803460 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78519647-bfc6-4f2f-a16f-b2109a5b31f5 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:16.803737 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78519647-bfc6-4f2f-a16f-b2109a5b31f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:16.804110 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78519647-bfc6-4f2f-a16f-b2109a5b31f5 None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:16.804838 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78519647-bfc6-4f2f-a16f-b2109a5b31f5 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:16.805138 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78519647-bfc6-4f2f-a16f-b2109a5b31f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:16.806559 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78519647-bfc6-4f2f-a16f-b2109a5b31f5 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:16.827002 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40469bf9-b8de-4abb-9c21-308f5a3128ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:16.827632 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40469bf9-b8de-4abb-9c21-308f5a3128ad None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:16.827912 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40469bf9-b8de-4abb-9c21-308f5a3128ad None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:16.828219 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40469bf9-b8de-4abb-9c21-308f5a3128ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:16.839202 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40469bf9-b8de-4abb-9c21-308f5a3128ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:16.840689 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40469bf9-b8de-4abb-9c21-308f5a3128ad None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:16.843261 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 201/390] 10.222.0.26 () {56 vars in 1137 bytes} [Wed Nov 2 03:21:16 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:16.843957 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78519647-bfc6-4f2f-a16f-b2109a5b31f5 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:16.856841 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-40469bf9-b8de-4abb-9c21-308f5a3128ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:16.856841 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:16.856841 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:16.856841 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:16.856841 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:16.856841 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:16.856841 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:16.856841 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:16.856841 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:16.856841 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:16.856841 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:16.856841 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:16.856841 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 03:21:16.856841 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 03:21:16.856841 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 03:21:16.856841 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 03:21:16.857826 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:16.857826 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:16.857826 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:16.857826 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:16.857826 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:16.857826 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:16.857826 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:16.857826 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:16.857826 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:16.857826 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:16.857826 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:16.857826 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:16.857826 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:16.857826 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:16.857826 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:16.857826 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:16.857826 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 03:21:16.858664 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 03:21:16.858664 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:16.858664 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:16.858664 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 190/391] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:21:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:16.859571 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40469bf9-b8de-4abb-9c21-308f5a3128ad None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:16.876756 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:16.945367 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78519647-bfc6-4f2f-a16f-b2109a5b31f5 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:16.963720 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40469bf9-b8de-4abb-9c21-308f5a3128ad None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:16.981285 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5224b71-7fc6-488c-a10e-5b5e1f0b1d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:16.981916 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5224b71-7fc6-488c-a10e-5b5e1f0b1d70 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:16.982198 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5224b71-7fc6-488c-a10e-5b5e1f0b1d70 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:16.982468 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5224b71-7fc6-488c-a10e-5b5e1f0b1d70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:16.992918 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5224b71-7fc6-488c-a10e-5b5e1f0b1d70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:16.994375 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5224b71-7fc6-488c-a10e-5b5e1f0b1d70 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:16.998015 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f63e2-8cb6-458d-bead-0691e6089dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:16.998699 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f63e2-8cb6-458d-bead-0691e6089dfc None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:16.998975 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f63e2-8cb6-458d-bead-0691e6089dfc None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:16.999245 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f63e2-8cb6-458d-bead-0691e6089dfc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:17.009410 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-c5224b71-7fc6-488c-a10e-5b5e1f0b1d70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:17.009410 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:17.009410 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:17.009410 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:17.009410 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:17.009410 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:17.009410 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:17.009410 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:17.009410 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:17.009410 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:17.009410 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:17.009410 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:17.009410 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 03:21:17.009410 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 03:21:17.009410 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 03:21:17.009410 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 03:21:17.010283 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:17.010283 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:17.010283 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:17.010283 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:17.010283 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:17.010283 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:17.010283 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:17.010283 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:17.010283 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:17.010283 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:17.010283 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:17.010283 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:17.010283 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:17.010283 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:17.010283 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:17.010283 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:17.010283 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 03:21:17.011132 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 03:21:17.011132 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:17.011132 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:17.011295 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562f63e2-8cb6-458d-bead-0691e6089dfc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:17.011372 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 202/392] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:21:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:17.011372 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5224b71-7fc6-488c-a10e-5b5e1f0b1d70 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:17.011490 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562f63e2-8cb6-458d-bead-0691e6089dfc None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:17.026724 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-562f63e2-8cb6-458d-bead-0691e6089dfc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:17.026724 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:17.026724 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:17.026724 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:17.026724 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:17.026724 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:17.026724 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:17.026724 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:17.026724 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:17.026724 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:17.026724 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:17.026724 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:17.026724 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 03:21:17.026724 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 03:21:17.026724 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 03:21:17.026724 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 03:21:17.027866 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:17.027866 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:17.027866 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:17.027866 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:17.027866 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:17.027866 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:17.027866 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:17.027866 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:17.027866 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:17.027866 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:17.027866 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:17.027866 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:17.027866 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:17.027866 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:17.027866 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:17.027866 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:17.027866 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 03:21:17.028753 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 03:21:17.028753 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:17.028753 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:17.028753 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 191/393] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:21:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:17.028753 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f63e2-8cb6-458d-bead-0691e6089dfc None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:17.059305 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:17.094752 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:17.117494 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5224b71-7fc6-488c-a10e-5b5e1f0b1d70 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:17.130476 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f63e2-8cb6-458d-bead-0691e6089dfc None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:17.152080 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cdca9f-03b7-4787-a299-316661a0a5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:17.152692 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cdca9f-03b7-4787-a299-316661a0a5b0 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:17.152971 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cdca9f-03b7-4787-a299-316661a0a5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:17.153245 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cdca9f-03b7-4787-a299-316661a0a5b0 None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:17.153971 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cdca9f-03b7-4787-a299-316661a0a5b0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:17.154274 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cdca9f-03b7-4787-a299-316661a0a5b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:17.155718 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cdca9f-03b7-4787-a299-316661a0a5b0 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:17.165131 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76522ec7-d848-41b0-9140-93f95050c947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:17.165768 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76522ec7-d848-41b0-9140-93f95050c947 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:17.166047 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76522ec7-d848-41b0-9140-93f95050c947 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:17.166321 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76522ec7-d848-41b0-9140-93f95050c947 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:17.172899 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 203/394] 10.222.0.26 () {56 vars in 1137 bytes} [Wed Nov 2 03:21:17 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:17.173736 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cdca9f-03b7-4787-a299-316661a0a5b0 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:17.177079 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76522ec7-d848-41b0-9140-93f95050c947 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:17.178619 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76522ec7-d848-41b0-9140-93f95050c947 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:17.193106 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-76522ec7-d848-41b0-9140-93f95050c947 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:17.193106 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:17.193106 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:17.193106 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:17.193106 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:17.193106 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:17.193106 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:17.193106 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:17.193106 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:17.193106 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:17.193106 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:17.193106 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:17.193106 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 03:21:17.193106 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 03:21:17.193106 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 03:21:17.193106 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 03:21:17.194090 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:17.194090 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:17.194090 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:17.194090 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:17.194090 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:17.194090 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:17.194090 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:17.194090 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:17.194090 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:17.194090 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:17.194090 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:17.194090 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:17.194090 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:17.194090 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:17.194090 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:17.194090 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:17.194090 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 03:21:17.195452 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 03:21:17.195452 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:17.195452 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:17.195452 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 192/395] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:21:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:17.195675 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76522ec7-d848-41b0-9140-93f95050c947 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:17.279533 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:17.282320 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cdca9f-03b7-4787-a299-316661a0a5b0 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:17.295233 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76522ec7-d848-41b0-9140-93f95050c947 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:17.312194 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:17.318264 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2317c10e-b842-4d7b-9648-618b47b29a52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:17.318882 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2317c10e-b842-4d7b-9648-618b47b29a52 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:17.319160 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2317c10e-b842-4d7b-9648-618b47b29a52 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:17.319435 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2317c10e-b842-4d7b-9648-618b47b29a52 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:17.320613 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2317c10e-b842-4d7b-9648-618b47b29a52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:17.320915 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2317c10e-b842-4d7b-9648-618b47b29a52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:17.322304 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2317c10e-b842-4d7b-9648-618b47b29a52 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:17.331879 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5072601b-0900-4ff8-b013-6ba97abe6ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:17.332568 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5072601b-0900-4ff8-b013-6ba97abe6ccf None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:17.332845 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5072601b-0900-4ff8-b013-6ba97abe6ccf None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:17.333117 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5072601b-0900-4ff8-b013-6ba97abe6ccf None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:17.333679 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 204/396] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Nov 2 03:21:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:17.333766 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5072601b-0900-4ff8-b013-6ba97abe6ccf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:17.334041 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5072601b-0900-4ff8-b013-6ba97abe6ccf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:17.334479 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2317c10e-b842-4d7b-9648-618b47b29a52 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:17.335455 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5072601b-0900-4ff8-b013-6ba97abe6ccf None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:17.347846 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 193/397] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Nov 2 03:21:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:17.348547 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5072601b-0900-4ff8-b013-6ba97abe6ccf None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:17.439207 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2317c10e-b842-4d7b-9648-618b47b29a52 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:17.448537 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5072601b-0900-4ff8-b013-6ba97abe6ccf None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:17.473585 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729866fe-d097-4ede-b3f4-2ff3e93247c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:17.474200 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729866fe-d097-4ede-b3f4-2ff3e93247c0 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:17.474505 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729866fe-d097-4ede-b3f4-2ff3e93247c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:17.474750 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729866fe-d097-4ede-b3f4-2ff3e93247c0 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:17.475407 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729866fe-d097-4ede-b3f4-2ff3e93247c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:17.475712 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729866fe-d097-4ede-b3f4-2ff3e93247c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:17.477679 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729866fe-d097-4ede-b3f4-2ff3e93247c0 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:17.483189 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5685b890-6a95-4709-9145-b69534ba6339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:17.483814 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685b890-6a95-4709-9145-b69534ba6339 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:17.484118 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685b890-6a95-4709-9145-b69534ba6339 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:17.484398 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5685b890-6a95-4709-9145-b69534ba6339 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:17.490338 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 205/398] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Nov 2 03:21:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:17.492274 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729866fe-d097-4ede-b3f4-2ff3e93247c0 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:17.499634 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5685b890-6a95-4709-9145-b69534ba6339 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:17.501135 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5685b890-6a95-4709-9145-b69534ba6339 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:17.514777 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:17.516195 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-5685b890-6a95-4709-9145-b69534ba6339 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:17.516195 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:17.516195 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:17.516195 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:17.516195 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:17.516195 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:17.516195 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:17.516195 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:17.516195 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:17.516195 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:17.516195 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:17.516195 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:17.516195 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 03:21:17.516195 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 03:21:17.516195 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 03:21:17.516195 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 03:21:17.517173 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:17.517173 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:17.517173 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:17.517173 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:17.517173 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:17.517173 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:17.517173 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:17.517173 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:17.517173 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:17.517173 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:17.517173 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:17.517173 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:17.517173 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:17.517173 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:17.517173 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:17.517173 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:17.517173 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 03:21:17.518030 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 03:21:17.518030 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:17.518030 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:17.518030 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 194/399] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:21:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:17.518256 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5685b890-6a95-4709-9145-b69534ba6339 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:17.548971 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:17.595049 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729866fe-d097-4ede-b3f4-2ff3e93247c0 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:17.618113 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5685b890-6a95-4709-9145-b69534ba6339 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:17.629350 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a392bc-a842-429d-8f62-a954acc95af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:17.629953 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a392bc-a842-429d-8f62-a954acc95af6 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:17.630232 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a392bc-a842-429d-8f62-a954acc95af6 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:17.630504 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a392bc-a842-429d-8f62-a954acc95af6 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:17.631144 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a392bc-a842-429d-8f62-a954acc95af6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:17.631447 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a392bc-a842-429d-8f62-a954acc95af6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:17.632910 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a392bc-a842-429d-8f62-a954acc95af6 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:17.644094 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 206/400] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Nov 2 03:21:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:17.644873 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a392bc-a842-429d-8f62-a954acc95af6 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:17.655588 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6876b11d-e3f4-4608-8529-6ee7421797f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:17.656249 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6876b11d-e3f4-4608-8529-6ee7421797f9 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:17.656532 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6876b11d-e3f4-4608-8529-6ee7421797f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:17.656812 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6876b11d-e3f4-4608-8529-6ee7421797f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:17.663492 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6876b11d-e3f4-4608-8529-6ee7421797f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:17.664962 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6876b11d-e3f4-4608-8529-6ee7421797f9 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:17.676426 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-6876b11d-e3f4-4608-8529-6ee7421797f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 03:21:17.676426 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:17.676426 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:17.676426 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:17.676426 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:17.676426 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:17.676426 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:17.676426 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:17.676426 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:17.676426 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:17.676426 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:17.676426 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:17.676426 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 03:21:17.676426 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 03:21:17.676426 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 03:21:17.677284 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 03:21:17.677284 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:17.677284 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:17.677284 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:17.677284 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:17.677284 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:17.677284 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:17.677284 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:17.677284 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:17.677284 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:17.677284 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:17.677284 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:17.677284 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:17.677284 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:17.677284 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:17.677284 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:17.677284 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:17.678294 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 03:21:17.678294 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 03:21:17.678294 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 03:21:17.678294 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 03:21:17.678294 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 03:21:17.678294 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 03:21:17.678294 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 03:21:17.678294 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:17.678294 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 195/401] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Nov 2 03:21:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:17.678294 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6876b11d-e3f4-4608-8529-6ee7421797f9 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:17.731256 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:17.750467 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a392bc-a842-429d-8f62-a954acc95af6 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:17.769952 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:17.780440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6876b11d-e3f4-4608-8529-6ee7421797f9 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:17.783336 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b2d01a-ff12-4d6f-92db-2ccbb5517c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:17.783936 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b2d01a-ff12-4d6f-92db-2ccbb5517c38 None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:17.784250 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b2d01a-ff12-4d6f-92db-2ccbb5517c38 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:17.784523 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b2d01a-ff12-4d6f-92db-2ccbb5517c38 None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:17.785160 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b2d01a-ff12-4d6f-92db-2ccbb5517c38 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:17.786584 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b2d01a-ff12-4d6f-92db-2ccbb5517c38 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:17.807131 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 207/402] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Nov 2 03:21:17 2022] POST /identity/v3/users => generated 247 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:21:17.807854 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b2d01a-ff12-4d6f-92db-2ccbb5517c38 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:17.820379 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a94a92-c027-44a1-8ef1-feebb642fe73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:17.821038 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a94a92-c027-44a1-8ef1-feebb642fe73 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:17.821317 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a94a92-c027-44a1-8ef1-feebb642fe73 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:17.821595 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a94a92-c027-44a1-8ef1-feebb642fe73 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:17.830587 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a94a92-c027-44a1-8ef1-feebb642fe73 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:17.832129 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a94a92-c027-44a1-8ef1-feebb642fe73 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:17.846206 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-46a94a92-c027-44a1-8ef1-feebb642fe73 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Nov 02 03:21:17.846206 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:17.846206 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:17.846206 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:17.846206 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:17.846206 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:17.846206 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:17.846206 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:17.846206 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:17.846206 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:17.846206 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:17.846206 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:17.846206 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 03:21:17.846206 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 03:21:17.846206 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 03:21:17.846206 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 03:21:17.847201 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:17.847201 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:17.847201 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 03:21:17.847201 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 03:21:17.847201 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 03:21:17.847201 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 03:21:17.847201 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:17.847201 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:17.847201 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:17.847201 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:17.847201 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:17.847201 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:17.847201 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:17.847201 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:17.847201 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:17.847201 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:17.847201 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:17.847201 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:17.848433 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:17.848433 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:17.848433 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 03:21:17.848433 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 03:21:17.848433 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 03:21:17.848433 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 03:21:17.848433 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 03:21:17.848433 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 03:21:17.848433 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Nov 02 03:21:17.848433 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:17.848433 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 196/403] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Nov 2 03:21:17 2022] GET /identity/v3/users/neutron => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:17.848433 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a94a92-c027-44a1-8ef1-feebb642fe73 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:17.932098 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b2d01a-ff12-4d6f-92db-2ccbb5517c38 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:17.964038 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a94a92-c027-44a1-8ef1-feebb642fe73 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:17.979290 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4259220d-159e-4460-afd5-c60389333d8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:17.979290 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259220d-159e-4460-afd5-c60389333d8d None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:17.979290 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259220d-159e-4460-afd5-c60389333d8d None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:17.979290 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4259220d-159e-4460-afd5-c60389333d8d None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:17.979855 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4259220d-159e-4460-afd5-c60389333d8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:17.980693 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4259220d-159e-4460-afd5-c60389333d8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:17.981443 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4259220d-159e-4460-afd5-c60389333d8d None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:17.984765 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:17.996506 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 208/404] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Nov 2 03:21:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:17.997382 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4259220d-159e-4460-afd5-c60389333d8d None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:18.010547 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52de715c-8ffd-4493-9423-43ac8faebd35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:18.011182 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de715c-8ffd-4493-9423-43ac8faebd35 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:18.011453 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de715c-8ffd-4493-9423-43ac8faebd35 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:18.012111 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de715c-8ffd-4493-9423-43ac8faebd35 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:18.019196 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52de715c-8ffd-4493-9423-43ac8faebd35 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:18.020666 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52de715c-8ffd-4493-9423-43ac8faebd35 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:18.033641 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-52de715c-8ffd-4493-9423-43ac8faebd35 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 03:21:18.033641 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:18.033641 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:18.033641 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:18.033641 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:18.033641 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:18.033641 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:18.033641 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:18.033641 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:18.033641 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:18.033641 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:18.033641 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:18.033641 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 03:21:18.033641 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 03:21:18.033641 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 03:21:18.034512 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 03:21:18.034512 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:18.034512 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:18.034512 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:18.034512 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:18.034512 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:18.034512 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:18.034512 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:18.034512 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:18.034512 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:18.034512 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:18.034512 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:18.034512 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:18.034512 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:18.034512 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:18.034512 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:18.034512 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:18.035366 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 03:21:18.035366 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 03:21:18.035366 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 03:21:18.035366 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 03:21:18.035366 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 03:21:18.035366 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 03:21:18.035366 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 03:21:18.035366 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:18.035366 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 197/405] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Nov 2 03:21:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:18.035848 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52de715c-8ffd-4493-9423-43ac8faebd35 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:18.048453 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:18.111107 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4259220d-159e-4460-afd5-c60389333d8d None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:18.139776 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52de715c-8ffd-4493-9423-43ac8faebd35 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:18.147679 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafffc2e-dddb-4c96-ba0d-2cd8d48b6ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:18.148326 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafffc2e-dddb-4c96-ba0d-2cd8d48b6ab5 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:18.148609 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafffc2e-dddb-4c96-ba0d-2cd8d48b6ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:18.149005 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafffc2e-dddb-4c96-ba0d-2cd8d48b6ab5 None admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:18.149696 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafffc2e-dddb-4c96-ba0d-2cd8d48b6ab5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:18.150002 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafffc2e-dddb-4c96-ba0d-2cd8d48b6ab5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:18.151569 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafffc2e-dddb-4c96-ba0d-2cd8d48b6ab5 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:18.174724 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 209/406] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Nov 2 03:21:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:18.174724 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafffc2e-dddb-4c96-ba0d-2cd8d48b6ab5 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:18.178238 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86189d9f-c11b-4d32-8cb8-a35fb0c55bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:18.179433 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86189d9f-c11b-4d32-8cb8-a35fb0c55bff None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:18.179789 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86189d9f-c11b-4d32-8cb8-a35fb0c55bff None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:18.180088 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86189d9f-c11b-4d32-8cb8-a35fb0c55bff None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:18.180822 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86189d9f-c11b-4d32-8cb8-a35fb0c55bff None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:18.181128 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86189d9f-c11b-4d32-8cb8-a35fb0c55bff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:18.182563 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86189d9f-c11b-4d32-8cb8-a35fb0c55bff None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:18.201764 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 198/407] 10.222.0.26 () {56 vars in 1139 bytes} [Wed Nov 2 03:21:18 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:18.202918 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86189d9f-c11b-4d32-8cb8-a35fb0c55bff None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:18.240764 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:18.277749 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:18.293206 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafffc2e-dddb-4c96-ba0d-2cd8d48b6ab5 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:18.313749 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86189d9f-c11b-4d32-8cb8-a35fb0c55bff None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:18.329881 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd1bf4b-8d84-4075-916c-1ea729e5b5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:18.330555 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd1bf4b-8d84-4075-916c-1ea729e5b5e8 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:18.330837 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd1bf4b-8d84-4075-916c-1ea729e5b5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:18.331111 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd1bf4b-8d84-4075-916c-1ea729e5b5e8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:18.338571 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd1bf4b-8d84-4075-916c-1ea729e5b5e8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:18.340180 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd1bf4b-8d84-4075-916c-1ea729e5b5e8 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:18.352298 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bd4725-09af-459d-a2c7-e4df54f797b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:18.352583 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-dcd1bf4b-8d84-4075-916c-1ea729e5b5e8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 03:21:18.352583 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:18.352583 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:18.352583 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:18.352583 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:18.352583 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:18.352583 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:18.352583 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:18.352583 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:18.352583 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:18.352583 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:18.352583 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:18.352583 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 03:21:18.352583 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 03:21:18.352583 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 03:21:18.353632 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 03:21:18.353632 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:18.353632 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:18.353632 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:18.353632 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:18.353632 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:18.353632 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:18.353632 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:18.353632 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:18.353632 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:18.353632 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:18.353632 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:18.353632 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:18.353632 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:18.353632 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:18.353632 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:18.353632 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:18.354768 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 03:21:18.354768 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 03:21:18.354768 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 03:21:18.354768 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 03:21:18.354768 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 03:21:18.354768 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 03:21:18.354768 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 03:21:18.354768 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:18.355275 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd4725-09af-459d-a2c7-e4df54f797b0 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:18.355275 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd4725-09af-459d-a2c7-e4df54f797b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:18.355275 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5bd4725-09af-459d-a2c7-e4df54f797b0 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:18.355849 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bd4725-09af-459d-a2c7-e4df54f797b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:18.356237 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bd4725-09af-459d-a2c7-e4df54f797b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:18.357726 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5bd4725-09af-459d-a2c7-e4df54f797b0 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:18.358513 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 210/408] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Nov 2 03:21:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:18.359324 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd1bf4b-8d84-4075-916c-1ea729e5b5e8 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:18.381048 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 199/409] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Nov 2 03:21:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:18.381048 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bd4725-09af-459d-a2c7-e4df54f797b0 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:18.479535 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd1bf4b-8d84-4075-916c-1ea729e5b5e8 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:18.484731 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5bd4725-09af-459d-a2c7-e4df54f797b0 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:18.496957 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:18.516936 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:18.517935 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce81d58d-7e38-485e-a426-e9ca6377763e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:18.519084 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81d58d-7e38-485e-a426-e9ca6377763e None admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:18.519602 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81d58d-7e38-485e-a426-e9ca6377763e None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:18.519835 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81d58d-7e38-485e-a426-e9ca6377763e None admin] PATH_INFO: `/v3/projects/d0b5e7017ac64ae6ab62c1a05aa54b72/users/44961b06151c4da382fcd54dd1f1e146/roles/e2a148d6fd534007bbb44d586d4306d1` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:18.529005 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc07d26-fd09-4119-b95e-0dd34b70114e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:18.529556 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc07d26-fd09-4119-b95e-0dd34b70114e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:18.529889 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc07d26-fd09-4119-b95e-0dd34b70114e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:18.530169 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc07d26-fd09-4119-b95e-0dd34b70114e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:18.536627 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce81d58d-7e38-485e-a426-e9ca6377763e None admin] RBAC: Authorizing `identity:create_grant(project_id=d0b5e7017ac64ae6ab62c1a05aa54b72, user_id=44961b06151c4da382fcd54dd1f1e146, role_id=e2a148d6fd534007bbb44d586d4306d1)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:18.538817 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce81d58d-7e38-485e-a426-e9ca6377763e None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:18.545194 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc07d26-fd09-4119-b95e-0dd34b70114e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:18.546910 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc07d26-fd09-4119-b95e-0dd34b70114e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:18.555008 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 211/410] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Nov 2 03:21:18 2022] PUT /identity/v3/projects/d0b5e7017ac64ae6ab62c1a05aa54b72/users/44961b06151c4da382fcd54dd1f1e146/roles/e2a148d6fd534007bbb44d586d4306d1 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:21:18.556220 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce81d58d-7e38-485e-a426-e9ca6377763e None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:18.564231 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-9dc07d26-fd09-4119-b95e-0dd34b70114e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:18.564231 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:18.564231 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:18.564231 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:18.564231 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:18.564231 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:18.564231 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:18.564231 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:18.564231 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:18.564231 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:18.564231 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:18.564231 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:18.564231 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 03:21:18.564231 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 03:21:18.564231 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 03:21:18.564231 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 03:21:18.565414 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:18.565414 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:18.565414 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:18.565414 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:18.565414 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:18.565414 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:18.565414 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:18.565414 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:18.565414 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:18.565414 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:18.565414 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:18.565414 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:18.565414 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:18.565414 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:18.565414 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:18.565414 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:18.565414 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 03:21:18.566294 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 03:21:18.566294 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:18.566294 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:18.566800 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 200/411] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:21:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:18.567696 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc07d26-fd09-4119-b95e-0dd34b70114e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:18.694036 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce81d58d-7e38-485e-a426-e9ca6377763e None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:18.708192 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc07d26-fd09-4119-b95e-0dd34b70114e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:18.745981 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567a4db2-a56b-4600-a563-a9a2f5a8ecab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:18.746190 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a4db2-a56b-4600-a563-a9a2f5a8ecab None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:18.746291 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a4db2-a56b-4600-a563-a9a2f5a8ecab None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:18.746558 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a4db2-a56b-4600-a563-a9a2f5a8ecab None admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:18.747264 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567a4db2-a56b-4600-a563-a9a2f5a8ecab None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:18.747569 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567a4db2-a56b-4600-a563-a9a2f5a8ecab None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:18.749113 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567a4db2-a56b-4600-a563-a9a2f5a8ecab None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:18.755989 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccac69b-78cc-432e-8250-bd8d6e76ddb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:18.756925 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccac69b-78cc-432e-8250-bd8d6e76ddb5 None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:18.757912 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccac69b-78cc-432e-8250-bd8d6e76ddb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:18.758254 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccac69b-78cc-432e-8250-bd8d6e76ddb5 None admin] PATH_INFO: `/v3/projects/d0b5e7017ac64ae6ab62c1a05aa54b72/users/df541db335eb42dc8b36cc858fa7c22f/roles/e2a148d6fd534007bbb44d586d4306d1` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:18.765155 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 212/412] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Nov 2 03:21:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:18.765155 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567a4db2-a56b-4600-a563-a9a2f5a8ecab None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:18.775511 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccac69b-78cc-432e-8250-bd8d6e76ddb5 None admin] RBAC: Authorizing `identity:create_grant(project_id=d0b5e7017ac64ae6ab62c1a05aa54b72, user_id=df541db335eb42dc8b36cc858fa7c22f, role_id=e2a148d6fd534007bbb44d586d4306d1)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:18.777284 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccac69b-78cc-432e-8250-bd8d6e76ddb5 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:18.797546 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 201/413] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Nov 2 03:21:18 2022] PUT /identity/v3/projects/d0b5e7017ac64ae6ab62c1a05aa54b72/users/df541db335eb42dc8b36cc858fa7c22f/roles/e2a148d6fd534007bbb44d586d4306d1 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:21:18.798595 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccac69b-78cc-432e-8250-bd8d6e76ddb5 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:18.804671 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:18.870599 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:18.961520 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567a4db2-a56b-4600-a563-a9a2f5a8ecab None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:19.005429 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccac69b-78cc-432e-8250-bd8d6e76ddb5 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:19.031975 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b0c55d-d8ee-43f7-8665-e210ffa72ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:19.031975 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b0c55d-d8ee-43f7-8665-e210ffa72ac2 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:19.031975 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b0c55d-d8ee-43f7-8665-e210ffa72ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:19.031975 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b0c55d-d8ee-43f7-8665-e210ffa72ac2 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:19.032381 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b0c55d-d8ee-43f7-8665-e210ffa72ac2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:19.032381 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b0c55d-d8ee-43f7-8665-e210ffa72ac2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:19.032992 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b0c55d-d8ee-43f7-8665-e210ffa72ac2 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:19.046766 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 213/414] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Nov 2 03:21:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:19.046940 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c65e139-0d13-4a49-ae24-d3b5606c2cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:19.047508 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c65e139-0d13-4a49-ae24-d3b5606c2cef None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:19.047786 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c65e139-0d13-4a49-ae24-d3b5606c2cef None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:19.048084 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c65e139-0d13-4a49-ae24-d3b5606c2cef None admin] PATH_INFO: `/v3/projects/d0b5e7017ac64ae6ab62c1a05aa54b72/users/50aab624ee9c4651a7cd42124bcc465f/roles/e2a148d6fd534007bbb44d586d4306d1` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:19.053383 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b0c55d-d8ee-43f7-8665-e210ffa72ac2 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:19.081305 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c65e139-0d13-4a49-ae24-d3b5606c2cef None admin] RBAC: Authorizing `identity:create_grant(project_id=d0b5e7017ac64ae6ab62c1a05aa54b72, user_id=50aab624ee9c4651a7cd42124bcc465f, role_id=e2a148d6fd534007bbb44d586d4306d1)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:19.082752 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c65e139-0d13-4a49-ae24-d3b5606c2cef None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:19.108353 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 202/415] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Nov 2 03:21:18 2022] PUT /identity/v3/projects/d0b5e7017ac64ae6ab62c1a05aa54b72/users/50aab624ee9c4651a7cd42124bcc465f/roles/e2a148d6fd534007bbb44d586d4306d1 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:21:19.179867 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:19.203969 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b0c55d-d8ee-43f7-8665-e210ffa72ac2 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:19.257379 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c423503-8227-4b86-98ab-75250bb0975c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:19.257943 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c423503-8227-4b86-98ab-75250bb0975c None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:19.258221 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c423503-8227-4b86-98ab-75250bb0975c None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:19.258495 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c423503-8227-4b86-98ab-75250bb0975c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:19.271522 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c423503-8227-4b86-98ab-75250bb0975c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:19.273074 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c423503-8227-4b86-98ab-75250bb0975c None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:19.285426 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-4c423503-8227-4b86-98ab-75250bb0975c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 03:21:19.285426 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:19.285426 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:19.285426 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:19.285426 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:19.285426 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:19.285426 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:19.285426 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:19.285426 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:19.285426 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:19.285426 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:19.285426 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:19.285426 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 03:21:19.285426 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 03:21:19.285426 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 03:21:19.286561 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 03:21:19.286561 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:19.286561 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:19.286561 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:19.286561 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:19.286561 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:19.286561 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:19.286561 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:19.286561 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:19.286561 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:19.286561 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:19.286561 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:19.286561 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:19.286561 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:19.286561 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:19.286561 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:19.286561 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:19.287630 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 03:21:19.287630 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 03:21:19.287630 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 03:21:19.287630 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 03:21:19.287630 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 03:21:19.287630 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 03:21:19.287630 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 03:21:19.287630 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:19.287630 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 214/416] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Nov 2 03:21:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:19.294383 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c65e139-0d13-4a49-ae24-d3b5606c2cef None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:19.380796 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:19.404463 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c65e139-0d13-4a49-ae24-d3b5606c2cef None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:19.439695 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e754f26-55dd-4f31-bfff-be65302e7e54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:19.440527 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e754f26-55dd-4f31-bfff-be65302e7e54 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:19.440902 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e754f26-55dd-4f31-bfff-be65302e7e54 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:19.441324 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e754f26-55dd-4f31-bfff-be65302e7e54 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:19.442110 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e754f26-55dd-4f31-bfff-be65302e7e54 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:19.442509 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e754f26-55dd-4f31-bfff-be65302e7e54 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:19.444081 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e754f26-55dd-4f31-bfff-be65302e7e54 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:19.457088 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 203/417] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Nov 2 03:21:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:19.465026 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c423503-8227-4b86-98ab-75250bb0975c None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:19.596996 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c423503-8227-4b86-98ab-75250bb0975c None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:19.639408 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84de5d0b-d864-4c7a-8fc2-5e2adecbe2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:19.640421 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de5d0b-d864-4c7a-8fc2-5e2adecbe2a9 None admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:19.640851 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de5d0b-d864-4c7a-8fc2-5e2adecbe2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:19.641301 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84de5d0b-d864-4c7a-8fc2-5e2adecbe2a9 None admin] PATH_INFO: `/v3/projects/d0b5e7017ac64ae6ab62c1a05aa54b72/users/f07e1b51b00f42b7b74c3689268635cb/roles/e2a148d6fd534007bbb44d586d4306d1` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:19.661357 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84de5d0b-d864-4c7a-8fc2-5e2adecbe2a9 None admin] RBAC: Authorizing `identity:create_grant(project_id=d0b5e7017ac64ae6ab62c1a05aa54b72, user_id=f07e1b51b00f42b7b74c3689268635cb, role_id=e2a148d6fd534007bbb44d586d4306d1)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:19.662796 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84de5d0b-d864-4c7a-8fc2-5e2adecbe2a9 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:19.666496 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:19.682920 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 215/418] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Nov 2 03:21:19 2022] PUT /identity/v3/projects/d0b5e7017ac64ae6ab62c1a05aa54b72/users/f07e1b51b00f42b7b74c3689268635cb/roles/e2a148d6fd534007bbb44d586d4306d1 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:21:20.105965 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4499ed42-b317-4cc1-8171-fa52e9051f80 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:20.106504 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4499ed42-b317-4cc1-8171-fa52e9051f80 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:20.106504 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4499ed42-b317-4cc1-8171-fa52e9051f80 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:20.108258 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 204/419] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:21:20.113809 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0162e6e-8948-42a0-bec9-14d22e7ba092 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:20.114098 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0162e6e-8948-42a0-bec9-14d22e7ba092 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:20.114375 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0162e6e-8948-42a0-bec9-14d22e7ba092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:20.140965 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-e0162e6e-8948-42a0-bec9-14d22e7ba092 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:20.142479 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e0162e6e-8948-42a0-bec9-14d22e7ba092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:20.230564 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 216/420] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 03:21:20.643735 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4499ed42-b317-4cc1-8171-fa52e9051f80 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:20.764136 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4499ed42-b317-4cc1-8171-fa52e9051f80 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:20.804663 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38744386-cf85-40f9-95cc-09e96214b4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:20.805221 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38744386-cf85-40f9-95cc-09e96214b4ee None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:20.805499 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38744386-cf85-40f9-95cc-09e96214b4ee None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:20.805772 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38744386-cf85-40f9-95cc-09e96214b4ee None admin] PATH_INFO: `/v3/roles/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:20.819093 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:20.825027 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38744386-cf85-40f9-95cc-09e96214b4ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:21:20.830431 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-38744386-cf85-40f9-95cc-09e96214b4ee None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 03:21:20.830431 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:20.830431 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:20.830431 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:20.830431 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:20.830431 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:20.830431 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:20.830431 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:20.830431 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:20.830431 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:20.830431 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:20.830431 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:20.830431 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:21:20.830431 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:21:20.830431 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:21:20.830431 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:21:20.831331 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:21:20.831331 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:21:20.831331 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:21:20.831331 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:21:20.831331 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:20.831331 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:20.831331 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:20.831331 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:20.831331 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:20.831331 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:20.831331 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:20.831331 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:20.831331 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:20.831331 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:20.831331 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:20.831331 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:20.832175 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:20.832175 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:20.832175 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:20.832175 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:20.832175 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:21:20.832175 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:21:20.832175 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:21:20.832175 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:21:20.832175 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:21:20.832175 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:21:20.832175 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 03:21:20.832175 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:20.833493 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 205/421] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Nov 2 03:21:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:20.839232 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0162e6e-8948-42a0-bec9-14d22e7ba092 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:20.988768 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0162e6e-8948-42a0-bec9-14d22e7ba092 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:21.029675 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:21.039079 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2753dde-48c2-403a-9e2f-f568384bd968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:21.039694 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2753dde-48c2-403a-9e2f-f568384bd968 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:21.039977 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2753dde-48c2-403a-9e2f-f568384bd968 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:21.040256 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2753dde-48c2-403a-9e2f-f568384bd968 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:21.040928 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2753dde-48c2-403a-9e2f-f568384bd968 None admin] RBAC: Adding query filter params (name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:21.041285 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2753dde-48c2-403a-9e2f-f568384bd968 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:21.042646 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2753dde-48c2-403a-9e2f-f568384bd968 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:21.054206 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 217/422] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Nov 2 03:21:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:21.059468 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38744386-cf85-40f9-95cc-09e96214b4ee None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:21.187185 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38744386-cf85-40f9-95cc-09e96214b4ee None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:21.223167 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494351e8-6dbb-41e7-8dc7-b3b95432b7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:21.223850 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494351e8-6dbb-41e7-8dc7-b3b95432b7c3 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:21.224149 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494351e8-6dbb-41e7-8dc7-b3b95432b7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:21.224418 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494351e8-6dbb-41e7-8dc7-b3b95432b7c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:21.236360 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494351e8-6dbb-41e7-8dc7-b3b95432b7c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:21.238061 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-494351e8-6dbb-41e7-8dc7-b3b95432b7c3 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:21.253122 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-494351e8-6dbb-41e7-8dc7-b3b95432b7c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:21.253122 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:21.253122 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:21.253122 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:21.253122 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:21.253122 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:21.253122 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:21.253122 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:21.253122 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:21.253122 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:21.253122 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:21.253122 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:21.253122 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 03:21:21.253122 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 03:21:21.253122 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 03:21:21.253122 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 03:21:21.254058 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:21.254058 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:21.254058 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:21.254058 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:21.254058 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:21.254058 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:21.254058 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:21.254058 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:21.254058 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:21.254058 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:21.254058 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:21.254058 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:21.254058 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:21.254058 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:21.254058 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:21.254058 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:21.254058 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 03:21:21.255307 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 03:21:21.255307 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:21.255307 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:21.255307 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 206/423] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:21:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:21.260134 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2753dde-48c2-403a-9e2f-f568384bd968 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:21.368285 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2753dde-48c2-403a-9e2f-f568384bd968 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:21.403857 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8391fbcb-ce6a-4e68-bb82-34765fff5762 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:21.404225 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8391fbcb-ce6a-4e68-bb82-34765fff5762 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:21.404431 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8391fbcb-ce6a-4e68-bb82-34765fff5762 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:21.405818 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 207/424] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 03:21:21.406151 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5070fe6a-173d-486b-945e-8f390b1ea33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:21.406772 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5070fe6a-173d-486b-945e-8f390b1ea33e None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:21.407045 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5070fe6a-173d-486b-945e-8f390b1ea33e None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:21.407313 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5070fe6a-173d-486b-945e-8f390b1ea33e None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:21.407975 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5070fe6a-173d-486b-945e-8f390b1ea33e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:21.408280 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5070fe6a-173d-486b-945e-8f390b1ea33e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:21.409704 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5070fe6a-173d-486b-945e-8f390b1ea33e None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:21.413245 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756eea9-6e4a-457d-b8ec-efd1f05df418 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:21.413762 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756eea9-6e4a-457d-b8ec-efd1f05df418 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:21.414247 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2756eea9-6e4a-457d-b8ec-efd1f05df418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:21.424221 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 218/425] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Nov 2 03:21:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:21.435501 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5070fe6a-173d-486b-945e-8f390b1ea33e None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:21.439610 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-2756eea9-6e4a-457d-b8ec-efd1f05df418 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:21.441014 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2756eea9-6e4a-457d-b8ec-efd1f05df418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:21.528797 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:21.573884 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5070fe6a-173d-486b-945e-8f390b1ea33e None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:21.577547 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:21.594004 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 208/426] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 03:21:21.635100 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6524dc0b-c1ee-4fda-94a6-74a764913464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:21.635797 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6524dc0b-c1ee-4fda-94a6-74a764913464 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:21.636173 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6524dc0b-c1ee-4fda-94a6-74a764913464 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:21.636561 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6524dc0b-c1ee-4fda-94a6-74a764913464 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:21.647896 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6524dc0b-c1ee-4fda-94a6-74a764913464 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:21.649553 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6524dc0b-c1ee-4fda-94a6-74a764913464 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:21.669968 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-6524dc0b-c1ee-4fda-94a6-74a764913464 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Nov 02 03:21:21.669968 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:21.669968 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:21.669968 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:21.669968 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:21.669968 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:21.669968 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:21.669968 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:21.669968 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:21.669968 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:21.669968 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:21.669968 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:21.669968 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 03:21:21.669968 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 03:21:21.669968 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 03:21:21.669968 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 03:21:21.670969 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:21.670969 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:21.670969 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 03:21:21.670969 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 03:21:21.670969 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 03:21:21.670969 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 03:21:21.670969 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:21.670969 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:21.670969 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:21.670969 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:21.670969 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:21.670969 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:21.670969 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:21.670969 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:21.670969 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:21.670969 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:21.670969 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:21.670969 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:21.672131 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:21.672131 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:21.672131 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 03:21:21.672131 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 03:21:21.672131 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 03:21:21.672131 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 03:21:21.672131 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 03:21:21.672131 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 03:21:21.672131 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Nov 02 03:21:21.672131 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:21.672131 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 219/427] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Nov 2 03:21:21 2022] GET /identity/v3/users/swift => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 03:21:21.681142 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2756eea9-6e4a-457d-b8ec-efd1f05df418 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:21.786586 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9cb14e-e764-488d-8eba-7b33786ae891 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:21.786906 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9cb14e-e764-488d-8eba-7b33786ae891 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:21.787167 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9cb14e-e764-488d-8eba-7b33786ae891 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:21.790169 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 220/428] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:21:21.798200 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb2b0d-ed6f-4a0a-8060-e369bfa31835 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:21.798200 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb2b0d-ed6f-4a0a-8060-e369bfa31835 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:21.798200 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb2b0d-ed6f-4a0a-8060-e369bfa31835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:21.825879 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-6fcb2b0d-ed6f-4a0a-8060-e369bfa31835 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:21.827411 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-6fcb2b0d-ed6f-4a0a-8060-e369bfa31835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:21.850449 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2756eea9-6e4a-457d-b8ec-efd1f05df418 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:21.897156 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625590a7-190e-4bd0-9a79-49805027125b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:21.898261 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625590a7-190e-4bd0-9a79-49805027125b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:21.898547 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625590a7-190e-4bd0-9a79-49805027125b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:21.898818 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625590a7-190e-4bd0-9a79-49805027125b None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:21.899964 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625590a7-190e-4bd0-9a79-49805027125b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:21.900276 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625590a7-190e-4bd0-9a79-49805027125b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:21.901763 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625590a7-190e-4bd0-9a79-49805027125b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:21.917866 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 221/429] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 03:21:21.925396 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50849aa-e4a6-4ac3-9b3d-cbcba3aa7f91 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:21.925922 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50849aa-e4a6-4ac3-9b3d-cbcba3aa7f91 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:21.926399 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50849aa-e4a6-4ac3-9b3d-cbcba3aa7f91 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:21.928160 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 209/431] 10.222.0.26 () {56 vars in 1135 bytes} [Wed Nov 2 03:21:21 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:21.928375 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 222/431] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:21:21.930775 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625590a7-190e-4bd0-9a79-49805027125b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:21.936178 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ffdaa-16b8-48ad-9910-7fe0e3d61d4e None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:21.936708 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ffdaa-16b8-48ad-9910-7fe0e3d61d4e None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:21.937203 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ffdaa-16b8-48ad-9910-7fe0e3d61d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:21.975267 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-5f9ffdaa-16b8-48ad-9910-7fe0e3d61d4e None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:21.976511 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-5f9ffdaa-16b8-48ad-9910-7fe0e3d61d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:21.995916 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:22.033652 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:22.074864 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625590a7-190e-4bd0-9a79-49805027125b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:22.083149 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 223/432] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 03:21:22.096004 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9ffdaa-16b8-48ad-9910-7fe0e3d61d4e None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:22.149446 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47be0a4b-6c43-4a72-9e63-bd260514cad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:22.149996 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be0a4b-6c43-4a72-9e63-bd260514cad6 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:22.152899 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be0a4b-6c43-4a72-9e63-bd260514cad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:22.153194 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be0a4b-6c43-4a72-9e63-bd260514cad6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:22.174022 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47be0a4b-6c43-4a72-9e63-bd260514cad6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:22.175478 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47be0a4b-6c43-4a72-9e63-bd260514cad6 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:22.208351 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-47be0a4b-6c43-4a72-9e63-bd260514cad6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:22.208351 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:22.208351 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:22.208351 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:22.208351 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:22.208351 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:22.208351 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:22.208351 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:22.208351 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:22.208351 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:22.208351 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:22.208351 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:22.208351 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 03:21:22.208351 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 03:21:22.208351 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 03:21:22.208351 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 03:21:22.209782 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:22.209782 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:22.209782 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:22.209782 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:22.209782 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:22.209782 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:22.209782 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:22.209782 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:22.209782 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:22.209782 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:22.209782 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:22.209782 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:22.209782 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:22.209782 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:22.209782 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:22.209782 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:22.209782 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 03:21:22.211161 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 03:21:22.211161 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:22.211161 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:22.212995 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 210/433] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:21:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 03:21:22.218567 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47be0a4b-6c43-4a72-9e63-bd260514cad6 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:22.253412 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9ffdaa-16b8-48ad-9910-7fe0e3d61d4e None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:22.291602 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ebf37e-4827-4281-a6ba-f5889f8a73cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:22.292676 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ebf37e-4827-4281-a6ba-f5889f8a73cf None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:22.293154 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ebf37e-4827-4281-a6ba-f5889f8a73cf None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:22.293636 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ebf37e-4827-4281-a6ba-f5889f8a73cf None admin] PATH_INFO: `/v3/roles/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:22.305507 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ebf37e-4827-4281-a6ba-f5889f8a73cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:21:22.311171 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-75ebf37e-4827-4281-a6ba-f5889f8a73cf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 03:21:22.311171 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:22.311171 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:22.311171 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:22.311171 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:22.311171 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:22.311171 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:22.311171 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:22.311171 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:22.311171 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:22.311171 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:22.311171 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:22.311171 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:21:22.311171 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:21:22.311171 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:21:22.311171 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:21:22.312301 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:21:22.312301 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:21:22.312301 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:21:22.312301 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:21:22.312301 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:22.312301 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:22.312301 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:22.312301 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:22.312301 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:22.312301 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:22.312301 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:22.312301 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:22.312301 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:22.312301 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:22.312301 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:22.312301 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:22.313097 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:22.313097 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:22.313097 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:22.313097 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:22.313097 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:21:22.313097 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:21:22.313097 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:21:22.313097 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:21:22.313097 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:21:22.313097 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:21:22.313097 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 03:21:22.313097 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:22.320176 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 224/434] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Nov 2 03:21:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:22.320176 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ebf37e-4827-4281-a6ba-f5889f8a73cf None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:22.337554 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:22.349863 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47be0a4b-6c43-4a72-9e63-bd260514cad6 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:22.361477 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:22.395513 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b51296-650b-45f2-b08c-9e9e93ec7ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:22.396152 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b51296-650b-45f2-b08c-9e9e93ec7ef1 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:22.396434 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b51296-650b-45f2-b08c-9e9e93ec7ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:22.396703 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b51296-650b-45f2-b08c-9e9e93ec7ef1 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:22.397343 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b51296-650b-45f2-b08c-9e9e93ec7ef1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:22.398223 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b51296-650b-45f2-b08c-9e9e93ec7ef1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:22.399637 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b51296-650b-45f2-b08c-9e9e93ec7ef1 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:22.411147 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 211/435] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Nov 2 03:21:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:22.411929 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b51296-650b-45f2-b08c-9e9e93ec7ef1 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:22.437774 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ebf37e-4827-4281-a6ba-f5889f8a73cf None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:22.471870 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991aa886-9df6-4644-88a4-ce1d2ac755c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:22.472523 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991aa886-9df6-4644-88a4-ce1d2ac755c0 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:22.472796 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991aa886-9df6-4644-88a4-ce1d2ac755c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:22.473062 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991aa886-9df6-4644-88a4-ce1d2ac755c0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:22.482900 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991aa886-9df6-4644-88a4-ce1d2ac755c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:21:22.487985 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-991aa886-9df6-4644-88a4-ce1d2ac755c0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 03:21:22.487985 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:22.487985 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:22.487985 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:22.487985 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:22.487985 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:22.487985 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:22.487985 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:22.487985 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:22.487985 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:22.487985 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:22.487985 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:22.487985 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:21:22.487985 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:21:22.487985 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:21:22.487985 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:21:22.488870 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:21:22.488870 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:21:22.488870 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:21:22.488870 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:21:22.488870 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:22.488870 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:22.488870 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:22.488870 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:22.488870 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:22.488870 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:22.488870 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:22.488870 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:22.488870 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:22.488870 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:22.488870 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:22.488870 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:22.489674 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:22.489674 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:22.489674 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:22.489674 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:22.489674 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:21:22.489674 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:21:22.489674 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:21:22.489674 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:21:22.489674 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:21:22.489674 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:21:22.489674 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 03:21:22.489674 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:22.489674 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 225/436] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Nov 2 03:21:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:22.490918 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991aa886-9df6-4644-88a4-ce1d2ac755c0 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:22.513060 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b51296-650b-45f2-b08c-9e9e93ec7ef1 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:22.551424 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af63f987-d151-4c9f-9658-2eee90af0bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:22.552090 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af63f987-d151-4c9f-9658-2eee90af0bd1 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:22.552367 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af63f987-d151-4c9f-9658-2eee90af0bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:22.552632 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af63f987-d151-4c9f-9658-2eee90af0bd1 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:22.553278 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af63f987-d151-4c9f-9658-2eee90af0bd1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:22.553572 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af63f987-d151-4c9f-9658-2eee90af0bd1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:22.555059 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af63f987-d151-4c9f-9658-2eee90af0bd1 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:22.562812 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 212/437] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Nov 2 03:21:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:22.563989 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2dd36b-43fd-4185-a4ba-ce7c963bdd94 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:22.564282 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2dd36b-43fd-4185-a4ba-ce7c963bdd94 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:22.564616 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2dd36b-43fd-4185-a4ba-ce7c963bdd94 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:22.565804 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 213/438] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:22 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:21:22.566494 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2dd36b-43fd-4185-a4ba-ce7c963bdd94 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:22.578160 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:22.582785 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:22.591647 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991aa886-9df6-4644-88a4-ce1d2ac755c0 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:22.626252 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e7373e-bf8b-4355-8916-b3f64c511528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:22.626871 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e7373e-bf8b-4355-8916-b3f64c511528 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:22.627152 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e7373e-bf8b-4355-8916-b3f64c511528 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:22.627431 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e7373e-bf8b-4355-8916-b3f64c511528 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:22.637804 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e7373e-bf8b-4355-8916-b3f64c511528 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:21:22.642607 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-c1e7373e-bf8b-4355-8916-b3f64c511528 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 03:21:22.642607 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:22.642607 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:22.642607 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:22.642607 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:22.642607 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:22.642607 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:22.642607 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:22.642607 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:22.642607 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:22.642607 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:22.642607 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:22.642607 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:21:22.642607 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:21:22.642607 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:21:22.642607 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:21:22.643700 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:21:22.643700 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:21:22.643700 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:21:22.643700 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:21:22.643700 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:22.643700 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:22.643700 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:22.643700 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:22.643700 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:22.643700 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:22.643700 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:22.643700 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:22.643700 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:22.643700 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:22.643700 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:22.643700 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:22.644535 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:22.644535 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:22.644535 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:22.644535 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:22.644535 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:21:22.644535 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:21:22.644535 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:21:22.644535 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:21:22.644535 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:21:22.644535 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:21:22.644535 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 03:21:22.644535 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:22.644535 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 226/439] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Nov 2 03:21:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:22.645216 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e7373e-bf8b-4355-8916-b3f64c511528 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:22.672083 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2dd36b-43fd-4185-a4ba-ce7c963bdd94 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:22.705507 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ac7ad1-b6cf-41d2-b7c4-98878802e44c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:22.706151 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac7ad1-b6cf-41d2-b7c4-98878802e44c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:22.706426 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac7ad1-b6cf-41d2-b7c4-98878802e44c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:22.706697 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ac7ad1-b6cf-41d2-b7c4-98878802e44c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:22.713864 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ac7ad1-b6cf-41d2-b7c4-98878802e44c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:22.715290 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ac7ad1-b6cf-41d2-b7c4-98878802e44c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:22.726743 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-43ac7ad1-b6cf-41d2-b7c4-98878802e44c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 03:21:22.726743 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:22.726743 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:22.726743 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:22.726743 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:22.726743 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:22.726743 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:22.726743 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:22.726743 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:22.726743 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:22.726743 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:22.726743 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:22.726743 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 03:21:22.726743 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 03:21:22.726743 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 03:21:22.727593 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 03:21:22.727593 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:22.727593 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:22.727593 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:22.727593 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:22.727593 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:22.727593 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:22.727593 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:22.727593 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:22.727593 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:22.727593 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:22.727593 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:22.727593 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:22.727593 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:22.727593 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:22.727593 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:22.727593 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:22.728478 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 03:21:22.728478 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 03:21:22.728478 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 03:21:22.728478 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 03:21:22.728478 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 03:21:22.728478 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 03:21:22.728478 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 03:21:22.728478 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:22.728478 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 214/440] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Nov 2 03:21:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:22.729003 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ac7ad1-b6cf-41d2-b7c4-98878802e44c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:22.745009 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e7373e-bf8b-4355-8916-b3f64c511528 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:22.778935 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23a9112-e08f-4d28-8001-2fecd362a923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:22.779529 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a9112-e08f-4d28-8001-2fecd362a923 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:22.779806 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a9112-e08f-4d28-8001-2fecd362a923 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:22.780179 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23a9112-e08f-4d28-8001-2fecd362a923 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:22.780836 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23a9112-e08f-4d28-8001-2fecd362a923 None admin] RBAC: Adding query filter params (name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:22.781132 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23a9112-e08f-4d28-8001-2fecd362a923 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:22.782481 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23a9112-e08f-4d28-8001-2fecd362a923 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:22.790645 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 227/441] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Nov 2 03:21:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:22.791748 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca1ad4-96f4-42cd-a095-f0c328f055b7 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:22.792060 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca1ad4-96f4-42cd-a095-f0c328f055b7 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:22.792330 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca1ad4-96f4-42cd-a095-f0c328f055b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:22.803355 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:22.809794 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-70ca1ad4-96f4-42cd-a095-f0c328f055b7 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:22.810983 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-70ca1ad4-96f4-42cd-a095-f0c328f055b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:22.814476 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:22.828812 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ac7ad1-b6cf-41d2-b7c4-98878802e44c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:22.861735 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b2f371-6d8c-4540-9bb2-7f854db8dd23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:22.862626 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2f371-6d8c-4540-9bb2-7f854db8dd23 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:22.862896 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2f371-6d8c-4540-9bb2-7f854db8dd23 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:22.863163 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b2f371-6d8c-4540-9bb2-7f854db8dd23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:22.874110 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b2f371-6d8c-4540-9bb2-7f854db8dd23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:22.875527 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b2f371-6d8c-4540-9bb2-7f854db8dd23 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:22.892107 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 228/442] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 03:21:22.892107 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ca1ad4-96f4-42cd-a095-f0c328f055b7 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:22.892349 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-09b2f371-6d8c-4540-9bb2-7f854db8dd23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:22.892349 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:22.892349 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:22.892349 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:22.892349 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:22.892349 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:22.892349 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:22.892349 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:22.892349 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:22.892349 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:22.892349 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:22.892349 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:22.892349 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 03:21:22.892349 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 03:21:22.892349 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 03:21:22.892349 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 03:21:22.893362 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:22.893362 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:22.893362 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:22.893362 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:22.893362 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:22.893362 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:22.893362 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:22.893362 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:22.893362 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:22.893362 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:22.893362 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:22.893362 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:22.893362 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:22.893362 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:22.893362 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:22.893362 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:22.893362 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 03:21:22.894419 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 03:21:22.894419 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:22.894419 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:22.894419 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 215/443] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:21:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:22.894419 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b2f371-6d8c-4540-9bb2-7f854db8dd23 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:22.988140 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ca1ad4-96f4-42cd-a095-f0c328f055b7 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:22.996146 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b2f371-6d8c-4540-9bb2-7f854db8dd23 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:23.023605 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:23.024550 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47edd1a-bb65-4d21-81dc-51888f37e9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:23.025165 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47edd1a-bb65-4d21-81dc-51888f37e9cf None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:23.025444 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47edd1a-bb65-4d21-81dc-51888f37e9cf None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:23.025768 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47edd1a-bb65-4d21-81dc-51888f37e9cf None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:23.026555 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47edd1a-bb65-4d21-81dc-51888f37e9cf None admin] RBAC: Adding query filter params (name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:23.026928 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47edd1a-bb65-4d21-81dc-51888f37e9cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:23.028383 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47edd1a-bb65-4d21-81dc-51888f37e9cf None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:23.034121 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2524a8bd-b112-42fa-beaa-12b4cd6b9849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:23.034736 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2524a8bd-b112-42fa-beaa-12b4cd6b9849 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:23.035010 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2524a8bd-b112-42fa-beaa-12b4cd6b9849 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:23.035285 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2524a8bd-b112-42fa-beaa-12b4cd6b9849 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:23.035993 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2524a8bd-b112-42fa-beaa-12b4cd6b9849 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:23.036297 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2524a8bd-b112-42fa-beaa-12b4cd6b9849 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:23.037709 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2524a8bd-b112-42fa-beaa-12b4cd6b9849 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:23.041964 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 229/444] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Nov 2 03:21:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:23.042817 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47edd1a-bb65-4d21-81dc-51888f37e9cf None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:23.045415 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:23.052069 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 216/445] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Nov 2 03:21:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:23.052895 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2524a8bd-b112-42fa-beaa-12b4cd6b9849 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:23.148335 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47edd1a-bb65-4d21-81dc-51888f37e9cf None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:23.160324 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2524a8bd-b112-42fa-beaa-12b4cd6b9849 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:23.182082 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19abb0f5-ae5c-413a-a79b-2a6c03fe4a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:23.182702 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19abb0f5-ae5c-413a-a79b-2a6c03fe4a3b None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:23.182976 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19abb0f5-ae5c-413a-a79b-2a6c03fe4a3b None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:23.183248 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19abb0f5-ae5c-413a-a79b-2a6c03fe4a3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:23.194025 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19abb0f5-ae5c-413a-a79b-2a6c03fe4a3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:23.195485 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19abb0f5-ae5c-413a-a79b-2a6c03fe4a3b None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:23.195826 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023fd067-5e2b-4e52-9752-ecf100bd15b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:23.196464 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023fd067-5e2b-4e52-9752-ecf100bd15b5 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:23.196743 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023fd067-5e2b-4e52-9752-ecf100bd15b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:23.197014 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023fd067-5e2b-4e52-9752-ecf100bd15b5 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:23.197645 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023fd067-5e2b-4e52-9752-ecf100bd15b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:23.197948 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023fd067-5e2b-4e52-9752-ecf100bd15b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:23.199397 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023fd067-5e2b-4e52-9752-ecf100bd15b5 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:23.210489 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-19abb0f5-ae5c-413a-a79b-2a6c03fe4a3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:23.210489 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:23.210489 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:23.210489 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:23.210489 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:23.210489 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:23.210489 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:23.210489 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:23.210489 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:23.210489 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:23.210489 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:23.210489 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:23.210489 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 03:21:23.210489 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 03:21:23.210489 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 03:21:23.210489 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 03:21:23.211426 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:23.211426 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:23.211426 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:23.211426 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:23.211426 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:23.211426 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:23.211426 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:23.211426 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:23.211426 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:23.211426 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:23.211426 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:23.211426 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:23.211426 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:23.211426 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:23.211426 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:23.211426 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:23.211426 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 03:21:23.212301 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 03:21:23.212301 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:23.212301 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:23.212480 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 217/446] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Nov 2 03:21:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:23.212480 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023fd067-5e2b-4e52-9752-ecf100bd15b5 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:23.215983 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 230/447] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:21:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:23.216783 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19abb0f5-ae5c-413a-a79b-2a6c03fe4a3b None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:23.260059 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:23.272611 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:23.312545 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023fd067-5e2b-4e52-9752-ecf100bd15b5 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:23.316858 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19abb0f5-ae5c-413a-a79b-2a6c03fe4a3b None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:23.345348 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e765c34-1986-4d99-84f7-57c303b5bbda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:23.345974 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e765c34-1986-4d99-84f7-57c303b5bbda None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:23.346259 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e765c34-1986-4d99-84f7-57c303b5bbda None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:23.346534 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e765c34-1986-4d99-84f7-57c303b5bbda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:23.352151 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccf8749-12d2-4b4d-9dfc-0b9c0ea5f936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:23.352796 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccf8749-12d2-4b4d-9dfc-0b9c0ea5f936 None admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:23.353076 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccf8749-12d2-4b4d-9dfc-0b9c0ea5f936 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:23.353348 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccf8749-12d2-4b4d-9dfc-0b9c0ea5f936 None admin] PATH_INFO: `/v3/projects/d0b5e7017ac64ae6ab62c1a05aa54b72/users/5193c40a917b4105ab8b27aead1685f8/roles/e2a148d6fd534007bbb44d586d4306d1` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:23.357386 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e765c34-1986-4d99-84f7-57c303b5bbda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:23.358824 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e765c34-1986-4d99-84f7-57c303b5bbda None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:23.368352 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccf8749-12d2-4b4d-9dfc-0b9c0ea5f936 None admin] RBAC: Authorizing `identity:create_grant(project_id=d0b5e7017ac64ae6ab62c1a05aa54b72, user_id=5193c40a917b4105ab8b27aead1685f8, role_id=e2a148d6fd534007bbb44d586d4306d1)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:23.369796 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccf8749-12d2-4b4d-9dfc-0b9c0ea5f936 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:23.373661 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-6e765c34-1986-4d99-84f7-57c303b5bbda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:23.373661 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:23.373661 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:23.373661 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:23.373661 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:23.373661 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:23.373661 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:23.373661 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:23.373661 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:23.373661 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:23.373661 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:23.373661 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:23.373661 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 03:21:23.373661 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 03:21:23.373661 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 03:21:23.373661 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 03:21:23.374981 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:23.374981 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:23.374981 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:23.374981 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:23.374981 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:23.374981 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:23.374981 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:23.374981 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:23.374981 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:23.374981 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:23.374981 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:23.374981 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:23.374981 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:23.374981 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:23.374981 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:23.374981 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:23.374981 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 03:21:23.376469 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 03:21:23.376469 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:23.376469 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:23.376469 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 218/448] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:21:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:23.376685 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e765c34-1986-4d99-84f7-57c303b5bbda None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:23.383010 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 231/449] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Nov 2 03:21:23 2022] PUT /identity/v3/projects/d0b5e7017ac64ae6ab62c1a05aa54b72/users/5193c40a917b4105ab8b27aead1685f8/roles/e2a148d6fd534007bbb44d586d4306d1 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:21:23.383814 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccf8749-12d2-4b4d-9dfc-0b9c0ea5f936 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:23.497624 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:23.498082 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccf8749-12d2-4b4d-9dfc-0b9c0ea5f936 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:23.501028 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:23.503252 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e765c34-1986-4d99-84f7-57c303b5bbda None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:23.544167 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993ac619-609a-4e86-beee-58feaec5c28e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:23.544832 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ac619-609a-4e86-beee-58feaec5c28e None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:23.545096 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ac619-609a-4e86-beee-58feaec5c28e None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:23.545436 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993ac619-609a-4e86-beee-58feaec5c28e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:23.550479 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19399092-6952-422e-bbd3-a478ff082686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:23.551126 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19399092-6952-422e-bbd3-a478ff082686 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:23.551406 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19399092-6952-422e-bbd3-a478ff082686 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:23.552326 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19399092-6952-422e-bbd3-a478ff082686 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:23.553545 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993ac619-609a-4e86-beee-58feaec5c28e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:23.555555 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993ac619-609a-4e86-beee-58feaec5c28e None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:23.565111 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19399092-6952-422e-bbd3-a478ff082686 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:21:23.570482 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-19399092-6952-422e-bbd3-a478ff082686 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 03:21:23.570482 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:23.570482 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:23.570482 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:23.570482 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:23.570482 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:23.570482 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:23.570482 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:23.570482 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:23.570482 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:23.570482 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:23.570482 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:23.570482 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:21:23.570482 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:21:23.570482 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:21:23.570482 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:21:23.571440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:21:23.571440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:21:23.571440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:21:23.571440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:21:23.571440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:23.571440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:23.571440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:23.571440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:23.571440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:23.571440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:23.571440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:23.571440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:23.571440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:23.571440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:23.571440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:23.571440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:23.572271 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:23.572271 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:23.572271 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:23.572271 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:23.572271 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:21:23.572271 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:21:23.572271 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:21:23.572271 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:21:23.572271 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:21:23.572271 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:21:23.572271 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 03:21:23.572271 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:23.572900 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-993ac619-609a-4e86-beee-58feaec5c28e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Nov 02 03:21:23.572900 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:23.572900 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:23.572900 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:23.572900 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:23.572900 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:23.572900 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:23.572900 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:23.572900 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:23.572900 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:23.572900 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:23.572900 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:23.572900 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 03:21:23.572900 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 03:21:23.572900 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 03:21:23.572900 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 03:21:23.573727 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:23.573727 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:23.573727 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 03:21:23.573727 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 03:21:23.573727 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 03:21:23.573727 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 03:21:23.573727 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:23.573727 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:23.573727 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:23.573727 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:23.573727 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:23.573727 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:23.573727 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:23.573727 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:23.573727 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:23.573727 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:23.573727 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:23.573727 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:23.574655 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:23.574655 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:23.574655 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 03:21:23.574655 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 03:21:23.574655 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 03:21:23.574655 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 03:21:23.574655 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 03:21:23.574655 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 03:21:23.574655 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Nov 02 03:21:23.574655 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:23.574655 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 232/451] 10.222.0.26 () {56 vars in 1093 bytes} [Wed Nov 2 03:21:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:23.575238 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 219/451] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Nov 2 03:21:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:23.575307 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993ac619-609a-4e86-beee-58feaec5c28e None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:23.577415 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19399092-6952-422e-bbd3-a478ff082686 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:23.672257 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993ac619-609a-4e86-beee-58feaec5c28e None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:23.685132 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19399092-6952-422e-bbd3-a478ff082686 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:23.704995 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d570fe3-3cdd-4093-ace8-aa35eba1f7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:23.705650 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d570fe3-3cdd-4093-ace8-aa35eba1f7fa None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:23.705929 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d570fe3-3cdd-4093-ace8-aa35eba1f7fa None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:23.706201 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d570fe3-3cdd-4093-ace8-aa35eba1f7fa None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:23.706836 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d570fe3-3cdd-4093-ace8-aa35eba1f7fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:23.707131 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d570fe3-3cdd-4093-ace8-aa35eba1f7fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:23.708523 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d570fe3-3cdd-4093-ace8-aa35eba1f7fa None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:23.720735 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 233/452] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Nov 2 03:21:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:23.721824 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d570fe3-3cdd-4093-ace8-aa35eba1f7fa None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:23.726266 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907abb47-f6ac-436d-aa38-5762fd65cbab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:23.726907 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907abb47-f6ac-436d-aa38-5762fd65cbab None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:23.727159 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907abb47-f6ac-436d-aa38-5762fd65cbab None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:23.727427 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907abb47-f6ac-436d-aa38-5762fd65cbab None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:23.728102 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907abb47-f6ac-436d-aa38-5762fd65cbab None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:23.728405 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907abb47-f6ac-436d-aa38-5762fd65cbab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:23.729764 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907abb47-f6ac-436d-aa38-5762fd65cbab None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:23.743125 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 220/453] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Nov 2 03:21:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:23.744237 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907abb47-f6ac-436d-aa38-5762fd65cbab None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:23.750854 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:23.767729 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:23.823901 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d570fe3-3cdd-4093-ace8-aa35eba1f7fa None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:23.857997 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907abb47-f6ac-436d-aa38-5762fd65cbab None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:23.861731 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b36b9c-3d7f-436d-ab4c-3b5377cd1c76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:23.862601 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b36b9c-3d7f-436d-ab4c-3b5377cd1c76 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:23.862885 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b36b9c-3d7f-436d-ab4c-3b5377cd1c76 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:23.863162 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b36b9c-3d7f-436d-ab4c-3b5377cd1c76 None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:23.863899 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b36b9c-3d7f-436d-ab4c-3b5377cd1c76 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:23.864240 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b36b9c-3d7f-436d-ab4c-3b5377cd1c76 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:23.865692 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b36b9c-3d7f-436d-ab4c-3b5377cd1c76 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:23.883581 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 234/454] 10.222.0.26 () {56 vars in 1133 bytes} [Wed Nov 2 03:21:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:23.884436 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b36b9c-3d7f-436d-ab4c-3b5377cd1c76 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:23.898829 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5db4bc-c3b7-4cf7-aacd-6d555342c484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:23.899668 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5db4bc-c3b7-4cf7-aacd-6d555342c484 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:23.900229 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5db4bc-c3b7-4cf7-aacd-6d555342c484 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:23.900795 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5db4bc-c3b7-4cf7-aacd-6d555342c484 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:23.901455 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5db4bc-c3b7-4cf7-aacd-6d555342c484 None admin] RBAC: Adding query filter params (name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:23.901759 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5db4bc-c3b7-4cf7-aacd-6d555342c484 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:23.903143 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5db4bc-c3b7-4cf7-aacd-6d555342c484 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:23.916770 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 221/455] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Nov 2 03:21:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:23.917490 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5db4bc-c3b7-4cf7-aacd-6d555342c484 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:23.968145 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:23.987988 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b36b9c-3d7f-436d-ab4c-3b5377cd1c76 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:24.004345 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:24.023631 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b5af60-d4f1-4c93-8542-fdb48c0ec214 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:24.024296 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b5af60-d4f1-4c93-8542-fdb48c0ec214 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:24.024596 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b5af60-d4f1-4c93-8542-fdb48c0ec214 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:24.024858 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b5af60-d4f1-4c93-8542-fdb48c0ec214 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:24.027011 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5db4bc-c3b7-4cf7-aacd-6d555342c484 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:24.031931 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b5af60-d4f1-4c93-8542-fdb48c0ec214 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:24.033399 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b5af60-d4f1-4c93-8542-fdb48c0ec214 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:24.045565 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-b6b5af60-d4f1-4c93-8542-fdb48c0ec214 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Nov 02 03:21:24.045565 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:24.045565 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:24.045565 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:24.045565 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:24.045565 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:24.045565 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:24.045565 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:24.045565 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:24.045565 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:24.045565 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:24.045565 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:24.045565 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 03:21:24.045565 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 03:21:24.045565 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 03:21:24.045565 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 03:21:24.046453 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:24.046453 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:24.046453 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 03:21:24.046453 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 03:21:24.046453 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 03:21:24.046453 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 03:21:24.046453 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:24.046453 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:24.046453 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:24.046453 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:24.046453 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:24.046453 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:24.046453 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:24.046453 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:24.046453 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:24.046453 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:24.046453 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:24.046453 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:24.047347 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:24.047347 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:24.047347 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 03:21:24.047347 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 03:21:24.047347 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 03:21:24.047347 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 03:21:24.047347 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 03:21:24.047347 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 03:21:24.047347 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Nov 02 03:21:24.047347 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:24.047347 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 235/456] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Nov 2 03:21:23 2022] GET /identity/v3/users/glance => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:24.047931 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b5af60-d4f1-4c93-8542-fdb48c0ec214 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:24.066524 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa408fc-b7df-4cd8-8f89-615a6e812382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:24.067488 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa408fc-b7df-4cd8-8f89-615a6e812382 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:24.067929 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa408fc-b7df-4cd8-8f89-615a6e812382 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:24.068368 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa408fc-b7df-4cd8-8f89-615a6e812382 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:24.076930 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa408fc-b7df-4cd8-8f89-615a6e812382 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:24.078569 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa408fc-b7df-4cd8-8f89-615a6e812382 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:24.090769 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-afa408fc-b7df-4cd8-8f89-615a6e812382 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Nov 02 03:21:24.090769 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:24.090769 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:24.090769 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:24.090769 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:24.090769 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:24.090769 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:24.090769 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:24.090769 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:24.090769 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:24.090769 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:24.090769 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:24.090769 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 03:21:24.090769 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 03:21:24.090769 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 03:21:24.090769 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 03:21:24.091689 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:24.091689 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:24.091689 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 03:21:24.091689 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 03:21:24.091689 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 03:21:24.091689 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 03:21:24.091689 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:24.091689 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:24.091689 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:24.091689 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:24.091689 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:24.091689 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:24.091689 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:24.091689 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:24.091689 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:24.091689 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:24.091689 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:24.091689 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:24.093254 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:24.093254 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:24.093254 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 03:21:24.093254 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 03:21:24.093254 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 03:21:24.093254 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 03:21:24.093254 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 03:21:24.093254 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 03:21:24.093254 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Nov 02 03:21:24.093254 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:24.093254 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 222/457] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Nov 2 03:21:23 2022] GET /identity/v3/users/cinder => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:24.093254 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa408fc-b7df-4cd8-8f89-615a6e812382 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:24.160626 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b5af60-d4f1-4c93-8542-fdb48c0ec214 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:24.194853 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a40396-0f69-4976-a82a-08ec081f8dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:24.195461 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a40396-0f69-4976-a82a-08ec081f8dfe None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:24.195745 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a40396-0f69-4976-a82a-08ec081f8dfe None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:24.196051 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a40396-0f69-4976-a82a-08ec081f8dfe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:24.198521 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa408fc-b7df-4cd8-8f89-615a6e812382 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:24.208689 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:24.234638 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed2708c-6925-485f-accf-60549aa1c20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:24.235287 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed2708c-6925-485f-accf-60549aa1c20c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:24.235536 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed2708c-6925-485f-accf-60549aa1c20c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:24.235801 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed2708c-6925-485f-accf-60549aa1c20c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:24.245428 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:24.248757 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed2708c-6925-485f-accf-60549aa1c20c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:24.250580 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed2708c-6925-485f-accf-60549aa1c20c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:24.267000 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-1ed2708c-6925-485f-accf-60549aa1c20c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:24.267000 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:24.267000 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:24.267000 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:24.267000 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:24.267000 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:24.267000 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:24.267000 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:24.267000 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:24.267000 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:24.267000 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:24.267000 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:24.267000 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 03:21:24.267000 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 03:21:24.267000 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 03:21:24.267000 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 03:21:24.268064 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:24.268064 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:24.268064 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:24.268064 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:24.268064 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:24.268064 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:24.268064 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:24.268064 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:24.268064 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:24.268064 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:24.268064 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:24.268064 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:24.268064 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:24.268064 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:24.268064 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:24.268064 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:24.268064 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 03:21:24.268933 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 03:21:24.268933 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:24.268933 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:24.268933 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 223/458] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:21:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:24.269149 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed2708c-6925-485f-accf-60549aa1c20c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:24.272395 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a40396-0f69-4976-a82a-08ec081f8dfe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:24.274396 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a40396-0f69-4976-a82a-08ec081f8dfe None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:24.375578 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed2708c-6925-485f-accf-60549aa1c20c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:24.380386 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 236/459] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Nov 2 03:21:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 03:21:24.380386 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a40396-0f69-4976-a82a-08ec081f8dfe None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:24.419675 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f655b666-8d7d-4122-8fee-33aa81977147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:24.420652 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f655b666-8d7d-4122-8fee-33aa81977147 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:24.420934 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f655b666-8d7d-4122-8fee-33aa81977147 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:24.421206 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f655b666-8d7d-4122-8fee-33aa81977147 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:24.421626 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:24.422559 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f655b666-8d7d-4122-8fee-33aa81977147 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:24.423152 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f655b666-8d7d-4122-8fee-33aa81977147 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:24.424607 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f655b666-8d7d-4122-8fee-33aa81977147 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:24.446493 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 224/460] 10.222.0.26 () {56 vars in 1137 bytes} [Wed Nov 2 03:21:24 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:24.447563 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f655b666-8d7d-4122-8fee-33aa81977147 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:24.480277 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a40396-0f69-4976-a82a-08ec081f8dfe None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:24.505855 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:24.515760 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc820d4-7065-484d-a6e1-1ef07ea37372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:24.516396 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc820d4-7065-484d-a6e1-1ef07ea37372 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:24.516670 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc820d4-7065-484d-a6e1-1ef07ea37372 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:24.516940 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc820d4-7065-484d-a6e1-1ef07ea37372 None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:24.517653 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc820d4-7065-484d-a6e1-1ef07ea37372 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:24.517954 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc820d4-7065-484d-a6e1-1ef07ea37372 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:24.519361 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc820d4-7065-484d-a6e1-1ef07ea37372 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:24.545395 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 237/461] 10.222.0.26 () {56 vars in 1137 bytes} [Wed Nov 2 03:21:24 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:24.545395 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc820d4-7065-484d-a6e1-1ef07ea37372 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:24.564405 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f655b666-8d7d-4122-8fee-33aa81977147 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:24.603812 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72a17ea-df3e-4cb9-b815-a6e93ba462c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:24.604621 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a17ea-df3e-4cb9-b815-a6e93ba462c5 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:24.604900 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a17ea-df3e-4cb9-b815-a6e93ba462c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:24.605260 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72a17ea-df3e-4cb9-b815-a6e93ba462c5 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:24.605802 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72a17ea-df3e-4cb9-b815-a6e93ba462c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:24.606102 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72a17ea-df3e-4cb9-b815-a6e93ba462c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:24.607469 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72a17ea-df3e-4cb9-b815-a6e93ba462c5 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:24.623584 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 225/462] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Nov 2 03:21:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:24.623584 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72a17ea-df3e-4cb9-b815-a6e93ba462c5 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:24.651752 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:24.655421 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc820d4-7065-484d-a6e1-1ef07ea37372 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:24.693536 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f090d759-29ef-4318-b1e0-c3c18b426b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:24.694169 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f090d759-29ef-4318-b1e0-c3c18b426b67 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:24.694967 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f090d759-29ef-4318-b1e0-c3c18b426b67 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:24.695242 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f090d759-29ef-4318-b1e0-c3c18b426b67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:24.705773 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f090d759-29ef-4318-b1e0-c3c18b426b67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:24.707238 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f090d759-29ef-4318-b1e0-c3c18b426b67 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:24.721968 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-f090d759-29ef-4318-b1e0-c3c18b426b67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:24.721968 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:24.721968 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:24.721968 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:24.721968 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:24.721968 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:24.721968 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:24.721968 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:24.721968 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:24.721968 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:24.721968 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:24.721968 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:24.721968 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 03:21:24.721968 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 03:21:24.721968 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 03:21:24.721968 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 03:21:24.722876 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:24.722876 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:24.722876 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:24.722876 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:24.722876 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:24.722876 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:24.722876 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:24.722876 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:24.722876 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:24.722876 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:24.722876 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:24.722876 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:24.722876 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:24.722876 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:24.722876 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:24.722876 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:24.722876 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 03:21:24.723710 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 03:21:24.723710 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:24.723710 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:24.723710 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 238/463] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:21:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:24.728696 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f090d759-29ef-4318-b1e0-c3c18b426b67 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:24.732052 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72a17ea-df3e-4cb9-b815-a6e93ba462c5 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:24.743483 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:24.770491 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c7a898-a1fe-41ce-8d5e-7e606e18fbf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:24.771319 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c7a898-a1fe-41ce-8d5e-7e606e18fbf6 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:24.771749 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c7a898-a1fe-41ce-8d5e-7e606e18fbf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:24.772079 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c7a898-a1fe-41ce-8d5e-7e606e18fbf6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:24.832915 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f090d759-29ef-4318-b1e0-c3c18b426b67 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:24.842450 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c7a898-a1fe-41ce-8d5e-7e606e18fbf6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:24.843986 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c7a898-a1fe-41ce-8d5e-7e606e18fbf6 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:24.868510 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65189559-9672-47ef-8056-f5a695cbbeaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:24.869113 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65189559-9672-47ef-8056-f5a695cbbeaa None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:24.869398 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65189559-9672-47ef-8056-f5a695cbbeaa None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:24.869675 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65189559-9672-47ef-8056-f5a695cbbeaa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:24.891232 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:24.939219 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65189559-9672-47ef-8056-f5a695cbbeaa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:24.940959 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65189559-9672-47ef-8056-f5a695cbbeaa None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:24.951198 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 226/464] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Nov 2 03:21:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 03:21:24.951198 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c7a898-a1fe-41ce-8d5e-7e606e18fbf6 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:24.961988 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:25.048848 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 239/465] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Nov 2 03:21:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 03:21:25.048848 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65189559-9672-47ef-8056-f5a695cbbeaa None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:25.066813 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c7a898-a1fe-41ce-8d5e-7e606e18fbf6 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:25.103355 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34736a28-9d10-452d-b5da-ce6459f0f688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:25.104027 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34736a28-9d10-452d-b5da-ce6459f0f688 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:25.104288 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34736a28-9d10-452d-b5da-ce6459f0f688 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:25.104561 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34736a28-9d10-452d-b5da-ce6459f0f688 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:25.108504 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:25.111984 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34736a28-9d10-452d-b5da-ce6459f0f688 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:25.113692 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34736a28-9d10-452d-b5da-ce6459f0f688 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:25.125256 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-34736a28-9d10-452d-b5da-ce6459f0f688 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Nov 02 03:21:25.125256 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:25.125256 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:25.125256 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:25.125256 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:25.125256 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:25.125256 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:25.125256 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:25.125256 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:25.125256 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:25.125256 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:25.125256 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:25.125256 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 03:21:25.125256 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 03:21:25.125256 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 03:21:25.125256 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 03:21:25.126173 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:25.126173 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:25.126173 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 03:21:25.126173 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 03:21:25.126173 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 03:21:25.126173 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 03:21:25.126173 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:25.126173 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:25.126173 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:25.126173 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:25.126173 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:25.126173 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:25.126173 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:25.126173 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:25.126173 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:25.126173 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:25.126173 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:25.126173 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:25.127080 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:25.127080 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:25.127080 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 03:21:25.127080 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 03:21:25.127080 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 03:21:25.127080 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 03:21:25.127080 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 03:21:25.127080 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 03:21:25.127080 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Nov 02 03:21:25.127080 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:25.127080 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 227/466] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Nov 2 03:21:24 2022] GET /identity/v3/users/neutron => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:25.127080 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34736a28-9d10-452d-b5da-ce6459f0f688 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:25.166139 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65189559-9672-47ef-8056-f5a695cbbeaa None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:25.202915 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aea47d-c92f-4ddd-8168-91b783eeee1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:25.203213 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:25.203659 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aea47d-c92f-4ddd-8168-91b783eeee1a None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:25.203884 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aea47d-c92f-4ddd-8168-91b783eeee1a None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:25.204448 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aea47d-c92f-4ddd-8168-91b783eeee1a None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:25.205806 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aea47d-c92f-4ddd-8168-91b783eeee1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:25.205806 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aea47d-c92f-4ddd-8168-91b783eeee1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:25.207725 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aea47d-c92f-4ddd-8168-91b783eeee1a None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:25.221204 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 240/467] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Nov 2 03:21:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:25.227042 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aea47d-c92f-4ddd-8168-91b783eeee1a None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:25.240829 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34736a28-9d10-452d-b5da-ce6459f0f688 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:25.278334 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449078c0-5110-4ee5-b212-1fa844e636ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:25.278977 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449078c0-5110-4ee5-b212-1fa844e636ed None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:25.279256 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449078c0-5110-4ee5-b212-1fa844e636ed None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:25.279530 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449078c0-5110-4ee5-b212-1fa844e636ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:25.290861 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449078c0-5110-4ee5-b212-1fa844e636ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:25.292872 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449078c0-5110-4ee5-b212-1fa844e636ed None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:25.308433 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-449078c0-5110-4ee5-b212-1fa844e636ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:25.308433 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:25.308433 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:25.308433 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:25.308433 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:25.308433 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:25.308433 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:25.308433 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:25.308433 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:25.308433 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:25.308433 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:25.308433 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:25.308433 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 03:21:25.308433 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 03:21:25.308433 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 03:21:25.308433 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 03:21:25.309536 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:25.309536 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:25.309536 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:25.309536 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:25.309536 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:25.309536 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:25.309536 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:25.309536 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:25.309536 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:25.309536 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:25.309536 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:25.309536 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:25.309536 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:25.309536 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:25.309536 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:25.309536 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:25.309536 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 03:21:25.310390 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 03:21:25.310390 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:25.310390 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:25.310390 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 228/468] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:21:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:25.310390 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449078c0-5110-4ee5-b212-1fa844e636ed None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:25.338848 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aea47d-c92f-4ddd-8168-91b783eeee1a None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:25.353451 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:25.376070 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394c4b5c-47a7-48f9-86bf-ff75acbf5a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:25.376699 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c4b5c-47a7-48f9-86bf-ff75acbf5a13 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:25.376979 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c4b5c-47a7-48f9-86bf-ff75acbf5a13 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:25.377252 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c4b5c-47a7-48f9-86bf-ff75acbf5a13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:25.389039 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394c4b5c-47a7-48f9-86bf-ff75acbf5a13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:25.390491 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394c4b5c-47a7-48f9-86bf-ff75acbf5a13 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:25.405282 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-394c4b5c-47a7-48f9-86bf-ff75acbf5a13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:25.405282 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:25.405282 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:25.405282 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:25.405282 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:25.405282 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:25.405282 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:25.405282 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:25.405282 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:25.405282 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:25.405282 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:25.405282 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:25.405282 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 03:21:25.405282 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 03:21:25.405282 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 03:21:25.405282 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 03:21:25.406314 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:25.406314 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:25.406314 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:25.406314 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:25.406314 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:25.406314 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:25.406314 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:25.406314 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:25.406314 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:25.406314 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:25.406314 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:25.406314 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:25.406314 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:25.406314 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:25.406314 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:25.406314 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:25.406314 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 03:21:25.407191 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 03:21:25.407191 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:25.407191 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:25.407191 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 241/469] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:21:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:25.407411 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394c4b5c-47a7-48f9-86bf-ff75acbf5a13 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:25.422848 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449078c0-5110-4ee5-b212-1fa844e636ed None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:25.447990 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:25.464601 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07c84e3-681f-4dce-bc1a-6e6684c9ad17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:25.465878 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c84e3-681f-4dce-bc1a-6e6684c9ad17 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:25.466338 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c84e3-681f-4dce-bc1a-6e6684c9ad17 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:25.466802 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c84e3-681f-4dce-bc1a-6e6684c9ad17 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:25.467705 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07c84e3-681f-4dce-bc1a-6e6684c9ad17 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:25.468227 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07c84e3-681f-4dce-bc1a-6e6684c9ad17 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:25.469810 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07c84e3-681f-4dce-bc1a-6e6684c9ad17 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:25.488677 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 229/470] 10.222.0.26 () {56 vars in 1139 bytes} [Wed Nov 2 03:21:25 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:25.489496 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07c84e3-681f-4dce-bc1a-6e6684c9ad17 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:25.519727 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394c4b5c-47a7-48f9-86bf-ff75acbf5a13 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:25.555191 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc68d8a2-fc1e-46cd-8705-8c6b479a9f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:25.555818 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc68d8a2-fc1e-46cd-8705-8c6b479a9f78 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:25.556129 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc68d8a2-fc1e-46cd-8705-8c6b479a9f78 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:25.556401 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc68d8a2-fc1e-46cd-8705-8c6b479a9f78 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:25.562854 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc68d8a2-fc1e-46cd-8705-8c6b479a9f78 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:25.564317 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc68d8a2-fc1e-46cd-8705-8c6b479a9f78 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:25.575328 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-dc68d8a2-fc1e-46cd-8705-8c6b479a9f78 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 03:21:25.575328 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:25.575328 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:25.575328 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:25.575328 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:25.575328 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:25.575328 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:25.575328 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:25.575328 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:25.575328 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:25.575328 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:25.575328 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:25.575328 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 03:21:25.575328 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 03:21:25.575328 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 03:21:25.576214 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 03:21:25.576214 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:25.576214 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:25.576214 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:25.576214 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:25.576214 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:25.576214 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:25.576214 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:25.576214 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:25.576214 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:25.576214 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:25.576214 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:25.576214 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:25.576214 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:25.576214 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:25.576214 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:25.576214 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:25.577067 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 03:21:25.577067 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 03:21:25.577067 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 03:21:25.577067 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 03:21:25.577067 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 03:21:25.577067 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 03:21:25.577067 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 03:21:25.577067 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:25.577067 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 242/471] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Nov 2 03:21:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:25.578584 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc68d8a2-fc1e-46cd-8705-8c6b479a9f78 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:25.591712 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07c84e3-681f-4dce-bc1a-6e6684c9ad17 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:25.603688 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:25.627122 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67b55d0-fc1f-432b-8d7b-b277e882349b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:25.627927 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b55d0-fc1f-432b-8d7b-b277e882349b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:25.628241 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b55d0-fc1f-432b-8d7b-b277e882349b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:25.628520 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67b55d0-fc1f-432b-8d7b-b277e882349b None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:25.629159 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67b55d0-fc1f-432b-8d7b-b277e882349b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:25.629469 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67b55d0-fc1f-432b-8d7b-b277e882349b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:25.630834 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67b55d0-fc1f-432b-8d7b-b277e882349b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:25.641661 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 230/472] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Nov 2 03:21:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:25.642445 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67b55d0-fc1f-432b-8d7b-b277e882349b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:25.680823 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc68d8a2-fc1e-46cd-8705-8c6b479a9f78 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:25.683110 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:25.714265 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1a2aa5-ecf4-4a92-b3b7-139a0a0c7ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:25.714918 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a2aa5-ecf4-4a92-b3b7-139a0a0c7ca9 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:25.715152 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a2aa5-ecf4-4a92-b3b7-139a0a0c7ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:25.715429 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1a2aa5-ecf4-4a92-b3b7-139a0a0c7ca9 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:25.716131 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1a2aa5-ecf4-4a92-b3b7-139a0a0c7ca9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:25.716438 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1a2aa5-ecf4-4a92-b3b7-139a0a0c7ca9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:25.717806 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1a2aa5-ecf4-4a92-b3b7-139a0a0c7ca9 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:25.729460 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 243/473] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Nov 2 03:21:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:25.730695 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c886db3b-23a1-4ba1-9c21-f8edeae4cede None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:25.730985 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c886db3b-23a1-4ba1-9c21-f8edeae4cede None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:25.731259 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c886db3b-23a1-4ba1-9c21-f8edeae4cede None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:25.732487 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 244/474] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:21:25.733173 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c886db3b-23a1-4ba1-9c21-f8edeae4cede None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:25.746631 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67b55d0-fc1f-432b-8d7b-b277e882349b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:25.780923 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefae6a5-902f-4168-a5f7-94e64dd3e8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:25.781527 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefae6a5-902f-4168-a5f7-94e64dd3e8da None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:25.781809 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefae6a5-902f-4168-a5f7-94e64dd3e8da None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:25.782085 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefae6a5-902f-4168-a5f7-94e64dd3e8da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:25.817228 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:25.833567 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c886db3b-23a1-4ba1-9c21-f8edeae4cede None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:25.848097 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefae6a5-902f-4168-a5f7-94e64dd3e8da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:25.849530 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefae6a5-902f-4168-a5f7-94e64dd3e8da None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:25.867727 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d3d2a7-7791-45fb-ba69-ffcab7c2aca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:25.868370 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d3d2a7-7791-45fb-ba69-ffcab7c2aca1 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:25.868650 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d3d2a7-7791-45fb-ba69-ffcab7c2aca1 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:25.868921 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d3d2a7-7791-45fb-ba69-ffcab7c2aca1 None admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:25.869604 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d3d2a7-7791-45fb-ba69-ffcab7c2aca1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:25.869909 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d3d2a7-7791-45fb-ba69-ffcab7c2aca1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:25.871314 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d3d2a7-7791-45fb-ba69-ffcab7c2aca1 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:25.887632 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 245/475] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Nov 2 03:21:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:25.887632 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d3d2a7-7791-45fb-ba69-ffcab7c2aca1 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:25.889495 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:25.947353 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 231/476] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Nov 2 03:21:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 03:21:25.948170 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefae6a5-902f-4168-a5f7-94e64dd3e8da None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:25.984344 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d3d2a7-7791-45fb-ba69-ffcab7c2aca1 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:26.017505 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736e93f9-d872-4bbe-b141-b4d080231da3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:26.018126 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736e93f9-d872-4bbe-b141-b4d080231da3 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:26.018406 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736e93f9-d872-4bbe-b141-b4d080231da3 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:26.018685 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736e93f9-d872-4bbe-b141-b4d080231da3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:26.025168 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736e93f9-d872-4bbe-b141-b4d080231da3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:26.026587 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736e93f9-d872-4bbe-b141-b4d080231da3 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:26.037595 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-736e93f9-d872-4bbe-b141-b4d080231da3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 03:21:26.037595 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:26.037595 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:26.037595 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:26.037595 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:26.037595 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:26.037595 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:26.037595 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:26.037595 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:26.037595 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:26.037595 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:26.037595 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:26.037595 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 03:21:26.037595 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 03:21:26.037595 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 03:21:26.039146 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 03:21:26.039146 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:26.039146 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:26.039146 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:26.039146 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:26.039146 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:26.039146 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:26.039146 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:26.039146 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:26.039146 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:26.039146 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:26.039146 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:26.039146 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:26.039146 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:26.039146 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:26.039146 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:26.039146 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:26.040239 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 03:21:26.040239 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 03:21:26.040239 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 03:21:26.040239 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 03:21:26.040239 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 03:21:26.040239 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 03:21:26.040239 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 03:21:26.040239 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:26.040239 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 246/477] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Nov 2 03:21:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:26.040808 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041df8b8-a489-4e34-89de-d5b174bdf7da None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:26.040808 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041df8b8-a489-4e34-89de-d5b174bdf7da None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:26.040959 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041df8b8-a489-4e34-89de-d5b174bdf7da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:26.047476 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:26.047769 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefae6a5-902f-4168-a5f7-94e64dd3e8da None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:26.058414 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-041df8b8-a489-4e34-89de-d5b174bdf7da None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:26.059590 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-041df8b8-a489-4e34-89de-d5b174bdf7da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:26.080882 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3bcbdf-d222-4d7f-8d35-b4be0d12e93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:26.081506 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3bcbdf-d222-4d7f-8d35-b4be0d12e93b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:26.081781 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3bcbdf-d222-4d7f-8d35-b4be0d12e93b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:26.082051 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3bcbdf-d222-4d7f-8d35-b4be0d12e93b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:26.088547 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3bcbdf-d222-4d7f-8d35-b4be0d12e93b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:26.089983 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3bcbdf-d222-4d7f-8d35-b4be0d12e93b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:26.092649 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:26.102322 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-5b3bcbdf-d222-4d7f-8d35-b4be0d12e93b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 03:21:26.102322 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:26.102322 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:26.102322 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:26.102322 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:26.102322 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:26.102322 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:26.102322 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:26.102322 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:26.102322 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:26.102322 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:26.102322 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:26.102322 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 03:21:26.102322 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 03:21:26.102322 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 03:21:26.103341 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 03:21:26.103341 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:26.103341 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:26.103341 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:26.103341 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:26.103341 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:26.103341 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:26.103341 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:26.103341 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:26.103341 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:26.103341 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:26.103341 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:26.103341 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:26.103341 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:26.103341 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:26.103341 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:26.103341 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:26.104339 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 03:21:26.104339 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 03:21:26.104339 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 03:21:26.104339 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 03:21:26.104339 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 03:21:26.104339 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 03:21:26.104339 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 03:21:26.104339 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:26.104339 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 232/478] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Nov 2 03:21:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:26.104339 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3bcbdf-d222-4d7f-8d35-b4be0d12e93b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:26.129648 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 247/479] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 03:21:26.130481 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041df8b8-a489-4e34-89de-d5b174bdf7da None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:26.209474 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3bcbdf-d222-4d7f-8d35-b4be0d12e93b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:26.235441 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041df8b8-a489-4e34-89de-d5b174bdf7da None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:26.245659 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ea7339-8cc7-419c-bcbc-411b93611b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:26.246264 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea7339-8cc7-419c-bcbc-411b93611b56 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:26.246538 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea7339-8cc7-419c-bcbc-411b93611b56 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:26.246806 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ea7339-8cc7-419c-bcbc-411b93611b56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:26.247575 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ea7339-8cc7-419c-bcbc-411b93611b56 None admin] RBAC: Adding query filter params (role.id=e2a148d6fd534007bbb44d586d4306d1, scope.project.id=d0b5e7017ac64ae6ab62c1a05aa54b72, user.id=44961b06151c4da382fcd54dd1f1e146) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:26.247879 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ea7339-8cc7-419c-bcbc-411b93611b56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:26.249336 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ea7339-8cc7-419c-bcbc-411b93611b56 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:26.255256 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 233/480] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Nov 2 03:21:26 2022] GET /identity/v3/role_assignments?user.id=44961b06151c4da382fcd54dd1f1e146&scope.project.id=d0b5e7017ac64ae6ab62c1a05aa54b72&role.id=e2a148d6fd534007bbb44d586d4306d1 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:26.255804 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ea7339-8cc7-419c-bcbc-411b93611b56 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:26.283552 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac5df63-5b77-4c66-a864-77e846e66ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:26.284267 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5df63-5b77-4c66-a864-77e846e66ed6 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:26.284543 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5df63-5b77-4c66-a864-77e846e66ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:26.284869 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac5df63-5b77-4c66-a864-77e846e66ed6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:26.287834 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:26.301263 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac5df63-5b77-4c66-a864-77e846e66ed6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:26.304095 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac5df63-5b77-4c66-a864-77e846e66ed6 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:26.326839 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-9ac5df63-5b77-4c66-a864-77e846e66ed6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:26.326839 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:26.326839 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:26.326839 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:26.326839 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:26.326839 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:26.326839 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:26.326839 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:26.326839 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:26.326839 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:26.326839 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:26.326839 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:26.326839 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 03:21:26.326839 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 03:21:26.326839 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 03:21:26.326839 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 03:21:26.327833 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:26.327833 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:26.327833 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:26.327833 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:26.327833 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:26.327833 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:26.327833 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:26.327833 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:26.327833 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:26.327833 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:26.327833 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:26.327833 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:26.327833 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:26.327833 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:26.327833 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:26.327833 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:26.327833 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 03:21:26.329326 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 03:21:26.329326 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:26.329326 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:26.329326 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 248/481] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:21:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:26.333080 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac5df63-5b77-4c66-a864-77e846e66ed6 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:26.345530 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:26.402251 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ea7339-8cc7-419c-bcbc-411b93611b56 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:26.441697 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730d006a-1901-4cfc-8c36-ab2830af7f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:26.442312 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d006a-1901-4cfc-8c36-ab2830af7f86 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:26.442771 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d006a-1901-4cfc-8c36-ab2830af7f86 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:26.443045 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d006a-1901-4cfc-8c36-ab2830af7f86 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:26.443739 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730d006a-1901-4cfc-8c36-ab2830af7f86 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:26.444072 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730d006a-1901-4cfc-8c36-ab2830af7f86 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:26.445572 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730d006a-1901-4cfc-8c36-ab2830af7f86 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:26.461669 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 234/482] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Nov 2 03:21:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:26.462494 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730d006a-1901-4cfc-8c36-ab2830af7f86 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:26.463854 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac5df63-5b77-4c66-a864-77e846e66ed6 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:26.500283 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b367aa41-f6d1-40dd-b09b-0126db121f6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:26.500814 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b367aa41-f6d1-40dd-b09b-0126db121f6e None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:26.501090 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b367aa41-f6d1-40dd-b09b-0126db121f6e None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:26.501360 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b367aa41-f6d1-40dd-b09b-0126db121f6e None admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:26.502043 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b367aa41-f6d1-40dd-b09b-0126db121f6e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:26.502340 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b367aa41-f6d1-40dd-b09b-0126db121f6e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:26.503750 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b367aa41-f6d1-40dd-b09b-0126db121f6e None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:26.515883 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 249/483] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Nov 2 03:21:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:26.516525 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b367aa41-f6d1-40dd-b09b-0126db121f6e None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:26.572497 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:26.577606 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730d006a-1901-4cfc-8c36-ab2830af7f86 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:26.603730 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:26.615764 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aecd853-0501-4dc9-b229-4b17e79e95b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:26.616446 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aecd853-0501-4dc9-b229-4b17e79e95b6 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:26.616726 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aecd853-0501-4dc9-b229-4b17e79e95b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:26.616992 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aecd853-0501-4dc9-b229-4b17e79e95b6 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:26.617641 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aecd853-0501-4dc9-b229-4b17e79e95b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:26.618184 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aecd853-0501-4dc9-b229-4b17e79e95b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:26.619305 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aecd853-0501-4dc9-b229-4b17e79e95b6 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:26.630998 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b367aa41-f6d1-40dd-b09b-0126db121f6e None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:26.635556 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 235/484] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Nov 2 03:21:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:26.635556 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aecd853-0501-4dc9-b229-4b17e79e95b6 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:26.671111 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7178ccc8-1593-493b-8ff8-ac1620a04213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:26.671745 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7178ccc8-1593-493b-8ff8-ac1620a04213 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:26.672050 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7178ccc8-1593-493b-8ff8-ac1620a04213 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:26.672323 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7178ccc8-1593-493b-8ff8-ac1620a04213 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:26.682708 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7178ccc8-1593-493b-8ff8-ac1620a04213 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:21:26.687462 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-7178ccc8-1593-493b-8ff8-ac1620a04213 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 03:21:26.687462 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:26.687462 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:26.687462 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:26.687462 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:26.687462 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:26.687462 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:26.687462 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:26.687462 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:26.687462 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:26.687462 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:26.687462 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:26.687462 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:21:26.687462 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:21:26.687462 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:21:26.687462 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:21:26.688406 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:21:26.688406 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:21:26.688406 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:21:26.688406 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:21:26.688406 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:26.688406 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:26.688406 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:26.688406 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:26.688406 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:26.688406 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:26.688406 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:26.688406 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:26.688406 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:26.688406 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:26.688406 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:26.688406 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:26.689305 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:26.689305 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:26.689305 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:26.689305 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:26.689305 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:21:26.689305 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:21:26.689305 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:21:26.689305 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:21:26.689305 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:21:26.689305 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:21:26.689305 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 03:21:26.689305 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:26.689305 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 250/485] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Nov 2 03:21:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:26.689974 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7178ccc8-1593-493b-8ff8-ac1620a04213 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:26.746502 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aecd853-0501-4dc9-b229-4b17e79e95b6 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:26.785135 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242fa868-6997-4535-87bb-25100af7a9eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:26.785818 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242fa868-6997-4535-87bb-25100af7a9eb None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:26.786182 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242fa868-6997-4535-87bb-25100af7a9eb None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:26.786535 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242fa868-6997-4535-87bb-25100af7a9eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:26.787318 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242fa868-6997-4535-87bb-25100af7a9eb None admin] RBAC: Adding query filter params (role.id=e2a148d6fd534007bbb44d586d4306d1, scope.project.id=d0b5e7017ac64ae6ab62c1a05aa54b72, user.id=50aab624ee9c4651a7cd42124bcc465f) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:26.787634 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242fa868-6997-4535-87bb-25100af7a9eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:26.789632 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242fa868-6997-4535-87bb-25100af7a9eb None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:26.795488 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 236/486] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Nov 2 03:21:26 2022] GET /identity/v3/role_assignments?user.id=50aab624ee9c4651a7cd42124bcc465f&scope.project.id=d0b5e7017ac64ae6ab62c1a05aa54b72&role.id=e2a148d6fd534007bbb44d586d4306d1 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:26.796316 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242fa868-6997-4535-87bb-25100af7a9eb None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:26.813894 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7178ccc8-1593-493b-8ff8-ac1620a04213 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:26.823842 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:26.857907 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:26.859233 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33f6004-1e87-49f5-aea1-d65ba6502880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:26.859934 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f6004-1e87-49f5-aea1-d65ba6502880 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:26.860583 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f6004-1e87-49f5-aea1-d65ba6502880 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:26.860583 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f6004-1e87-49f5-aea1-d65ba6502880 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:26.861645 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33f6004-1e87-49f5-aea1-d65ba6502880 None admin] RBAC: Adding query filter params (role.id=e2a148d6fd534007bbb44d586d4306d1, scope.project.id=d0b5e7017ac64ae6ab62c1a05aa54b72, user.id=df541db335eb42dc8b36cc858fa7c22f) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:26.862029 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33f6004-1e87-49f5-aea1-d65ba6502880 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:26.863869 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33f6004-1e87-49f5-aea1-d65ba6502880 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:26.876506 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 251/487] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Nov 2 03:21:26 2022] GET /identity/v3/role_assignments?user.id=df541db335eb42dc8b36cc858fa7c22f&scope.project.id=d0b5e7017ac64ae6ab62c1a05aa54b72&role.id=e2a148d6fd534007bbb44d586d4306d1 => generated 608 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:26.877730 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33f6004-1e87-49f5-aea1-d65ba6502880 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:26.925606 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242fa868-6997-4535-87bb-25100af7a9eb None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:26.967545 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7896b962-f3c2-4651-90dd-d00b454b768f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:26.968602 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7896b962-f3c2-4651-90dd-d00b454b768f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:26.969057 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7896b962-f3c2-4651-90dd-d00b454b768f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:26.969503 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7896b962-f3c2-4651-90dd-d00b454b768f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:26.978407 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7896b962-f3c2-4651-90dd-d00b454b768f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:26.986704 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7896b962-f3c2-4651-90dd-d00b454b768f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:27.007528 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-7896b962-f3c2-4651-90dd-d00b454b768f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 03:21:27.007528 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:27.007528 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:27.007528 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:27.007528 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:27.007528 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:27.007528 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:27.007528 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:27.007528 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:27.007528 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:27.007528 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:27.007528 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:27.007528 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 03:21:27.007528 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 03:21:27.007528 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 03:21:27.008467 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 03:21:27.008467 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:27.008467 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:27.008467 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:27.008467 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:27.008467 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:27.008467 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:27.008467 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:27.008467 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:27.008467 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:27.008467 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:27.008467 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:27.008467 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:27.008467 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:27.008467 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:27.008467 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:27.008467 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:27.009308 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 03:21:27.009308 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 03:21:27.009308 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 03:21:27.009308 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 03:21:27.009308 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 03:21:27.009308 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 03:21:27.009308 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 03:21:27.009308 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:27.009308 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 237/488] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Nov 2 03:21:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:27.017159 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7896b962-f3c2-4651-90dd-d00b454b768f None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:27.076619 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33f6004-1e87-49f5-aea1-d65ba6502880 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:27.123411 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a6325e-9515-4b20-ab4b-2367f8d1d913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:27.124108 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a6325e-9515-4b20-ab4b-2367f8d1d913 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:27.124392 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a6325e-9515-4b20-ab4b-2367f8d1d913 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:27.124671 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a6325e-9515-4b20-ab4b-2367f8d1d913 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:27.125356 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a6325e-9515-4b20-ab4b-2367f8d1d913 None admin] RBAC: Adding query filter params (name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:27.125662 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a6325e-9515-4b20-ab4b-2367f8d1d913 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:27.127874 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a6325e-9515-4b20-ab4b-2367f8d1d913 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:27.141310 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 252/489] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Nov 2 03:21:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:27.148893 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a6325e-9515-4b20-ab4b-2367f8d1d913 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:27.162938 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7896b962-f3c2-4651-90dd-d00b454b768f None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:27.175272 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:27.190354 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:27.208950 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d1cd78-86f8-4db7-89a9-1bf7ca2a6e5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:27.209867 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d1cd78-86f8-4db7-89a9-1bf7ca2a6e5a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:27.210812 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d1cd78-86f8-4db7-89a9-1bf7ca2a6e5a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:27.211236 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d1cd78-86f8-4db7-89a9-1bf7ca2a6e5a None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:27.211985 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d1cd78-86f8-4db7-89a9-1bf7ca2a6e5a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:27.212734 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d1cd78-86f8-4db7-89a9-1bf7ca2a6e5a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:27.214244 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d1cd78-86f8-4db7-89a9-1bf7ca2a6e5a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:27.228762 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 238/490] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Nov 2 03:21:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:27.234461 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d1cd78-86f8-4db7-89a9-1bf7ca2a6e5a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:27.287471 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a6325e-9515-4b20-ab4b-2367f8d1d913 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:27.353142 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d6ca7c-e3e3-4d8c-aad1-3a0b6673c40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:27.353700 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d6ca7c-e3e3-4d8c-aad1-3a0b6673c40a None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:27.353978 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d6ca7c-e3e3-4d8c-aad1-3a0b6673c40a None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:27.354248 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d6ca7c-e3e3-4d8c-aad1-3a0b6673c40a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:27.375064 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d1cd78-86f8-4db7-89a9-1bf7ca2a6e5a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:27.377498 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d6ca7c-e3e3-4d8c-aad1-3a0b6673c40a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:27.378961 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d6ca7c-e3e3-4d8c-aad1-3a0b6673c40a None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:27.407072 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-47d6ca7c-e3e3-4d8c-aad1-3a0b6673c40a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:27.407072 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:27.407072 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:27.407072 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:27.407072 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:27.407072 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:27.407072 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:27.407072 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:27.407072 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:27.407072 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:27.407072 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:27.407072 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:27.407072 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 03:21:27.407072 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 03:21:27.407072 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 03:21:27.407072 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 03:21:27.408058 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:27.408058 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:27.408058 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:27.408058 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:27.408058 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:27.408058 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:27.408058 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:27.408058 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:27.408058 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:27.408058 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:27.408058 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:27.408058 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:27.408058 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:27.408058 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:27.408058 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:27.408058 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:27.408058 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 03:21:27.408902 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 03:21:27.408902 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:27.408902 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:27.414375 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 253/491] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:21:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 03:21:27.415222 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d6ca7c-e3e3-4d8c-aad1-3a0b6673c40a None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:27.445934 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b3a8f4-281c-4872-b82c-d8f7139aa674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:27.446552 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3a8f4-281c-4872-b82c-d8f7139aa674 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:27.447800 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3a8f4-281c-4872-b82c-d8f7139aa674 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:27.448101 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b3a8f4-281c-4872-b82c-d8f7139aa674 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:27.448833 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b3a8f4-281c-4872-b82c-d8f7139aa674 None admin] RBAC: Adding query filter params (role.id=e2a148d6fd534007bbb44d586d4306d1, scope.project.id=d0b5e7017ac64ae6ab62c1a05aa54b72, user.id=f07e1b51b00f42b7b74c3689268635cb) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:27.450702 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b3a8f4-281c-4872-b82c-d8f7139aa674 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:27.452107 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b3a8f4-281c-4872-b82c-d8f7139aa674 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:27.469843 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 239/492] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Nov 2 03:21:27 2022] GET /identity/v3/role_assignments?user.id=f07e1b51b00f42b7b74c3689268635cb&scope.project.id=d0b5e7017ac64ae6ab62c1a05aa54b72&role.id=e2a148d6fd534007bbb44d586d4306d1 => generated 608 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:21:27.564375 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:27.586964 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d6ca7c-e3e3-4d8c-aad1-3a0b6673c40a None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:27.632355 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78637f1a-8d2a-4b79-946e-aaee6c665faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:27.632909 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78637f1a-8d2a-4b79-946e-aaee6c665faa None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:27.633187 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78637f1a-8d2a-4b79-946e-aaee6c665faa None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:27.633458 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78637f1a-8d2a-4b79-946e-aaee6c665faa None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:27.634117 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78637f1a-8d2a-4b79-946e-aaee6c665faa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:27.634418 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78637f1a-8d2a-4b79-946e-aaee6c665faa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:27.635801 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78637f1a-8d2a-4b79-946e-aaee6c665faa None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:27.654054 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 254/493] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Nov 2 03:21:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:27.659412 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b3a8f4-281c-4872-b82c-d8f7139aa674 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:27.708611 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:27.801709 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b3a8f4-281c-4872-b82c-d8f7139aa674 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:27.848484 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9015a6-8c87-480b-8c6d-5327db93a93a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:27.849127 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9015a6-8c87-480b-8c6d-5327db93a93a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:27.849407 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9015a6-8c87-480b-8c6d-5327db93a93a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:27.849681 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9015a6-8c87-480b-8c6d-5327db93a93a None admin] PATH_INFO: `/v3/users/swift` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:27.863165 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9015a6-8c87-480b-8c6d-5327db93a93a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:27.864690 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9015a6-8c87-480b-8c6d-5327db93a93a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:27.882043 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-0b9015a6-8c87-480b-8c6d-5327db93a93a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Nov 02 03:21:27.882043 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:27.882043 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:27.882043 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:27.882043 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:27.882043 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:27.882043 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:27.882043 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:27.882043 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:27.882043 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:27.882043 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:27.882043 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:27.882043 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 03:21:27.882043 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 03:21:27.882043 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 03:21:27.882043 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 03:21:27.883057 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:27.883057 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:27.883057 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 03:21:27.883057 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 03:21:27.883057 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 03:21:27.883057 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 03:21:27.883057 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:27.883057 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:27.883057 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:27.883057 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:27.883057 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:27.883057 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:27.883057 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:27.883057 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:27.883057 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:27.883057 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:27.883057 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:27.883057 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:27.884042 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:27.884042 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:27.884042 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 03:21:27.884042 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 03:21:27.884042 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 03:21:27.884042 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 03:21:27.884042 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 03:21:27.884042 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 03:21:27.884042 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Nov 02 03:21:27.884042 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:27.884042 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 240/494] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Nov 2 03:21:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:27.891357 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78637f1a-8d2a-4b79-946e-aaee6c665faa None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:28.004028 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78637f1a-8d2a-4b79-946e-aaee6c665faa None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:28.042692 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7d1c29-111e-43d0-a955-6ec753080fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:28.043310 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7d1c29-111e-43d0-a955-6ec753080fe4 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:28.043654 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7d1c29-111e-43d0-a955-6ec753080fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:28.043937 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7d1c29-111e-43d0-a955-6ec753080fe4 None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:28.044688 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7d1c29-111e-43d0-a955-6ec753080fe4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:28.044990 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7d1c29-111e-43d0-a955-6ec753080fe4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:28.046416 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7d1c29-111e-43d0-a955-6ec753080fe4 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:28.069788 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 255/495] 10.222.0.26 () {56 vars in 1135 bytes} [Wed Nov 2 03:21:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:28.071181 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9015a6-8c87-480b-8c6d-5327db93a93a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:28.185046 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:28.200411 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9015a6-8c87-480b-8c6d-5327db93a93a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:28.244085 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c3ab43-f468-43b3-80be-896e42a3a9ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:28.244778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c3ab43-f468-43b3-80be-896e42a3a9ea None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:28.245052 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c3ab43-f468-43b3-80be-896e42a3a9ea None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:28.245323 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c3ab43-f468-43b3-80be-896e42a3a9ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:28.329951 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c3ab43-f468-43b3-80be-896e42a3a9ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:28.331397 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c3ab43-f468-43b3-80be-896e42a3a9ea None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:28.431781 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:28.448704 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 241/496] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Nov 2 03:21:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Nov 02 03:21:28.457156 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7d1c29-111e-43d0-a955-6ec753080fe4 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:28.460476 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:28.571342 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e12e7-4c51-4bfa-8366-2f7be6314931 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:28.571654 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e12e7-4c51-4bfa-8366-2f7be6314931 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:28.571904 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192e12e7-4c51-4bfa-8366-2f7be6314931 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:28.573543 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 242/497] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:21:28.578338 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c13a9-ee8a-4e1d-9c25-2a76839f1221 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:28.578626 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c13a9-ee8a-4e1d-9c25-2a76839f1221 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:28.578903 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c13a9-ee8a-4e1d-9c25-2a76839f1221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:28.599086 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7d1c29-111e-43d0-a955-6ec753080fe4 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:28.613327 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-5a2c13a9-ee8a-4e1d-9c25-2a76839f1221 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:28.614619 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5a2c13a9-ee8a-4e1d-9c25-2a76839f1221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:28.634398 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b103619-cb09-4ff4-8ed9-adb1fa8ec5a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:28.635035 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b103619-cb09-4ff4-8ed9-adb1fa8ec5a3 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:28.635314 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b103619-cb09-4ff4-8ed9-adb1fa8ec5a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:28.635583 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b103619-cb09-4ff4-8ed9-adb1fa8ec5a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:28.647021 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b103619-cb09-4ff4-8ed9-adb1fa8ec5a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:28.649029 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b103619-cb09-4ff4-8ed9-adb1fa8ec5a3 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:28.666026 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-5b103619-cb09-4ff4-8ed9-adb1fa8ec5a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:28.666026 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:28.666026 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:28.666026 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:28.666026 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:28.666026 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:28.666026 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:28.666026 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:28.666026 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:28.666026 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:28.666026 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:28.666026 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:28.666026 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 03:21:28.666026 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 03:21:28.666026 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 03:21:28.666026 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 03:21:28.666978 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:28.666978 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:28.666978 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:28.666978 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:28.666978 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:28.666978 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:28.666978 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:28.666978 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:28.666978 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:28.666978 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:28.666978 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:28.666978 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:28.666978 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:28.666978 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:28.666978 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:28.666978 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:28.666978 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 03:21:28.667848 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 03:21:28.667848 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:28.667848 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:28.671878 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 256/498] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:21:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:28.673034 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b103619-cb09-4ff4-8ed9-adb1fa8ec5a3 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:28.735791 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:28.764613 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 243/499] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 03:21:28.817689 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b103619-cb09-4ff4-8ed9-adb1fa8ec5a3 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:28.857154 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7711ed3-a658-45e4-a07f-ba022ac70f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:28.857764 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7711ed3-a658-45e4-a07f-ba022ac70f4e None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:28.858038 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7711ed3-a658-45e4-a07f-ba022ac70f4e None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:28.858307 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7711ed3-a658-45e4-a07f-ba022ac70f4e None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:28.858945 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7711ed3-a658-45e4-a07f-ba022ac70f4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:28.859243 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7711ed3-a658-45e4-a07f-ba022ac70f4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:28.861230 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7711ed3-a658-45e4-a07f-ba022ac70f4e None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:28.880384 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 257/500] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Nov 2 03:21:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:21:28.885685 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2c13a9-ee8a-4e1d-9c25-2a76839f1221 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:29.004577 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2c13a9-ee8a-4e1d-9c25-2a76839f1221 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:29.028479 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:29.039035 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd7a63e-0a3e-4678-87d5-24476c8bc12a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:29.040134 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd7a63e-0a3e-4678-87d5-24476c8bc12a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:29.040861 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd7a63e-0a3e-4678-87d5-24476c8bc12a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:29.041130 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd7a63e-0a3e-4678-87d5-24476c8bc12a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:29.047811 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd7a63e-0a3e-4678-87d5-24476c8bc12a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:29.049273 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd7a63e-0a3e-4678-87d5-24476c8bc12a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:29.060616 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-bbd7a63e-0a3e-4678-87d5-24476c8bc12a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 03:21:29.060616 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:29.060616 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:29.060616 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:29.060616 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:29.060616 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:29.060616 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:29.060616 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:29.060616 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:29.060616 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:29.060616 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:29.060616 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:29.060616 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 03:21:29.060616 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 03:21:29.060616 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 03:21:29.061490 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 03:21:29.061490 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:29.061490 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:29.061490 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:29.061490 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:29.061490 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:29.061490 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:29.061490 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:29.061490 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:29.061490 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:29.061490 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:29.061490 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:29.061490 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:29.061490 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:29.061490 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:29.061490 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:29.061490 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:29.062853 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 03:21:29.062853 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 03:21:29.062853 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 03:21:29.062853 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 03:21:29.062853 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 03:21:29.062853 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 03:21:29.062853 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 03:21:29.062853 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:29.062853 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 244/501] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Nov 2 03:21:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:29.069010 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7711ed3-a658-45e4-a07f-ba022ac70f4e None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:29.089965 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd7a63e-0a3e-4678-87d5-24476c8bc12a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:29.245766 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7711ed3-a658-45e4-a07f-ba022ac70f4e None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:29.269078 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:29.276995 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd7a63e-0a3e-4678-87d5-24476c8bc12a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:29.308412 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98728585-8621-4861-a842-a642f6616cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:29.309380 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98728585-8621-4861-a842-a642f6616cbd None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:29.309863 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98728585-8621-4861-a842-a642f6616cbd None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:29.310334 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98728585-8621-4861-a842-a642f6616cbd None admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:29.311361 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98728585-8621-4861-a842-a642f6616cbd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:29.311986 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98728585-8621-4861-a842-a642f6616cbd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:29.313858 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98728585-8621-4861-a842-a642f6616cbd None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:29.324645 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30530b8a-4657-4a57-8d66-9c9368b6196b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:29.325297 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30530b8a-4657-4a57-8d66-9c9368b6196b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:29.325576 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30530b8a-4657-4a57-8d66-9c9368b6196b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:29.325844 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30530b8a-4657-4a57-8d66-9c9368b6196b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:29.328608 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 258/502] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Nov 2 03:21:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:29.334210 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98728585-8621-4861-a842-a642f6616cbd None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:29.338540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30530b8a-4657-4a57-8d66-9c9368b6196b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:21:29.343655 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-30530b8a-4657-4a57-8d66-9c9368b6196b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 03:21:29.343655 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:29.343655 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:29.343655 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:29.343655 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:29.343655 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:29.343655 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:29.343655 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:29.343655 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:29.343655 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:29.343655 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:29.343655 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:29.343655 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:21:29.343655 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:21:29.343655 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:21:29.343655 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:21:29.344616 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:21:29.344616 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:21:29.344616 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:21:29.344616 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:21:29.344616 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:29.344616 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:29.344616 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:29.344616 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:29.344616 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:29.344616 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:29.344616 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:29.344616 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:29.344616 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:29.344616 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:29.344616 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:29.344616 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:29.345531 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:29.345531 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:29.345531 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:29.345531 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:29.345531 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:21:29.345531 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:21:29.345531 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:21:29.345531 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:21:29.345531 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:21:29.345531 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:21:29.345531 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 03:21:29.345531 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:29.352279 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 245/503] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Nov 2 03:21:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 03:21:29.354626 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30530b8a-4657-4a57-8d66-9c9368b6196b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:29.401429 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:29.462199 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30530b8a-4657-4a57-8d66-9c9368b6196b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:29.500012 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad479c4b-f139-4e02-aef5-3c59ffe5ed86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:29.501014 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad479c4b-f139-4e02-aef5-3c59ffe5ed86 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:29.501497 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad479c4b-f139-4e02-aef5-3c59ffe5ed86 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:29.502000 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad479c4b-f139-4e02-aef5-3c59ffe5ed86 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:29.502900 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad479c4b-f139-4e02-aef5-3c59ffe5ed86 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:29.503417 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad479c4b-f139-4e02-aef5-3c59ffe5ed86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:29.505280 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad479c4b-f139-4e02-aef5-3c59ffe5ed86 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:29.515383 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 246/504] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Nov 2 03:21:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:29.521268 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad479c4b-f139-4e02-aef5-3c59ffe5ed86 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:29.629163 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad479c4b-f139-4e02-aef5-3c59ffe5ed86 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:29.633366 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:29.641248 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98728585-8621-4861-a842-a642f6616cbd None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:29.667441 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a04863b-f14e-4b9d-873c-15010a8528a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:29.668424 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a04863b-f14e-4b9d-873c-15010a8528a6 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:29.668923 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a04863b-f14e-4b9d-873c-15010a8528a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:29.669412 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a04863b-f14e-4b9d-873c-15010a8528a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:29.682652 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a04863b-f14e-4b9d-873c-15010a8528a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:29.684135 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a04863b-f14e-4b9d-873c-15010a8528a6 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:29.688703 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8f760-033e-4bd1-b2d7-7dfeaba0ee1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:29.689612 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8f760-033e-4bd1-b2d7-7dfeaba0ee1a None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:29.690120 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8f760-033e-4bd1-b2d7-7dfeaba0ee1a None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:29.690594 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8f760-033e-4bd1-b2d7-7dfeaba0ee1a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:29.692121 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f8f760-033e-4bd1-b2d7-7dfeaba0ee1a None admin] RBAC: Adding query filter params (role.id=e2a148d6fd534007bbb44d586d4306d1, scope.project.id=d0b5e7017ac64ae6ab62c1a05aa54b72, user.id=5193c40a917b4105ab8b27aead1685f8) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:29.692639 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f8f760-033e-4bd1-b2d7-7dfeaba0ee1a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:29.695067 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f8f760-033e-4bd1-b2d7-7dfeaba0ee1a None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:29.699400 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-4a04863b-f14e-4b9d-873c-15010a8528a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:29.699400 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:29.699400 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:29.699400 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:29.699400 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:29.699400 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:29.699400 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:29.699400 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:29.699400 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:29.699400 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:29.699400 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:29.699400 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:29.699400 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 03:21:29.699400 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 03:21:29.699400 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 03:21:29.699400 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 03:21:29.700354 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:29.700354 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:29.700354 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:29.700354 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:29.700354 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:29.700354 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:29.700354 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:29.700354 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:29.700354 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:29.700354 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:29.700354 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:29.700354 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:29.700354 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:29.700354 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:29.700354 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:29.700354 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:29.700354 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 03:21:29.701647 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 03:21:29.701647 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:29.701647 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:29.701647 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 247/505] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:21:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 03:21:29.708836 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92375468-831e-4fe2-8e82-748d3b92c867 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:29.708836 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92375468-831e-4fe2-8e82-748d3b92c867 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:29.708836 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92375468-831e-4fe2-8e82-748d3b92c867 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:29.708836 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 248/507] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:29 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:21:29.709424 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 259/507] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Nov 2 03:21:29 2022] GET /identity/v3/role_assignments?user.id=5193c40a917b4105ab8b27aead1685f8&scope.project.id=d0b5e7017ac64ae6ab62c1a05aa54b72&role.id=e2a148d6fd534007bbb44d586d4306d1 => generated 608 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:29.711981 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92375468-831e-4fe2-8e82-748d3b92c867 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:29.729212 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3fb7e-1d24-473f-9e6c-09629e1c6410 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:29.729848 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3fb7e-1d24-473f-9e6c-09629e1c6410 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:29.730334 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf3fb7e-1d24-473f-9e6c-09629e1c6410 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:29.773760 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-4bf3fb7e-1d24-473f-9e6c-09629e1c6410 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:29.777021 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-4bf3fb7e-1d24-473f-9e6c-09629e1c6410 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:29.845189 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:29.892836 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 260/508] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 03:21:29.906565 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be298c-6637-43d4-8cbe-afea71eec645 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:29.906870 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be298c-6637-43d4-8cbe-afea71eec645 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:29.907348 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be298c-6637-43d4-8cbe-afea71eec645 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:29.908724 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 261/509] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:21:29.915250 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8503e9e9-71ca-4aa0-a7bf-bdef8b683bba None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:29.915531 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8503e9e9-71ca-4aa0-a7bf-bdef8b683bba None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:29.915804 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8503e9e9-71ca-4aa0-a7bf-bdef8b683bba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:29.937077 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92375468-831e-4fe2-8e82-748d3b92c867 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:29.942300 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-8503e9e9-71ca-4aa0-a7bf-bdef8b683bba None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:29.943769 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-8503e9e9-71ca-4aa0-a7bf-bdef8b683bba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:29.985231 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c2629e-0252-46fa-b2ea-f84a9ca6849b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:29.985854 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c2629e-0252-46fa-b2ea-f84a9ca6849b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:29.986134 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c2629e-0252-46fa-b2ea-f84a9ca6849b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:29.986416 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c2629e-0252-46fa-b2ea-f84a9ca6849b None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:29.987731 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c2629e-0252-46fa-b2ea-f84a9ca6849b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:29.988089 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c2629e-0252-46fa-b2ea-f84a9ca6849b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:29.989772 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c2629e-0252-46fa-b2ea-f84a9ca6849b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:30.007713 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 249/510] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Nov 2 03:21:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:30.012666 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c2629e-0252-46fa-b2ea-f84a9ca6849b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:30.042153 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 262/511] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 03:21:30.064285 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:30.127399 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c2629e-0252-46fa-b2ea-f84a9ca6849b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:30.163231 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2540e8ef-460b-4606-ba21-defdbb3bba7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:30.163868 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2540e8ef-460b-4606-ba21-defdbb3bba7c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:30.164174 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2540e8ef-460b-4606-ba21-defdbb3bba7c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:30.164444 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2540e8ef-460b-4606-ba21-defdbb3bba7c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:30.171794 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2540e8ef-460b-4606-ba21-defdbb3bba7c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:30.173280 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2540e8ef-460b-4606-ba21-defdbb3bba7c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:30.182757 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8503e9e9-71ca-4aa0-a7bf-bdef8b683bba None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:30.187271 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-2540e8ef-460b-4606-ba21-defdbb3bba7c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Nov 02 03:21:30.187271 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:30.187271 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:30.187271 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:30.187271 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:30.187271 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:30.187271 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:30.187271 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:30.187271 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:30.187271 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:30.187271 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:30.187271 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:30.187271 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 03:21:30.187271 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 03:21:30.187271 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 03:21:30.187271 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 03:21:30.188227 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:30.188227 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:30.188227 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 03:21:30.188227 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 03:21:30.188227 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 03:21:30.188227 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 03:21:30.188227 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:30.188227 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:30.188227 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:30.188227 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:30.188227 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:30.188227 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:30.188227 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:30.188227 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:30.188227 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:30.188227 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:30.188227 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:30.188227 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:30.189238 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:30.189238 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:30.189238 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 03:21:30.189238 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 03:21:30.189238 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 03:21:30.189238 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 03:21:30.189238 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 03:21:30.189238 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 03:21:30.189238 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Nov 02 03:21:30.189238 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:30.189238 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 250/512] 10.222.0.26 () {56 vars in 1093 bytes} [Wed Nov 2 03:21:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:30.197155 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2540e8ef-460b-4606-ba21-defdbb3bba7c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:30.284202 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:30.311255 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2540e8ef-460b-4606-ba21-defdbb3bba7c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:30.346109 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:30.350378 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8503e9e9-71ca-4aa0-a7bf-bdef8b683bba None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:30.356021 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd14807-ca70-429a-8b49-eadfcd55a664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:30.356696 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd14807-ca70-429a-8b49-eadfcd55a664 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:30.356979 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd14807-ca70-429a-8b49-eadfcd55a664 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:30.357252 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd14807-ca70-429a-8b49-eadfcd55a664 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:30.357982 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd14807-ca70-429a-8b49-eadfcd55a664 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:30.358282 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd14807-ca70-429a-8b49-eadfcd55a664 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:30.360334 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd14807-ca70-429a-8b49-eadfcd55a664 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:30.383013 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 251/513] 10.222.0.26 () {56 vars in 1133 bytes} [Wed Nov 2 03:21:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:21:30.384464 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a232f9-31a8-4df3-9a13-893e7c04a2b1 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:30.384860 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a232f9-31a8-4df3-9a13-893e7c04a2b1 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:30.385236 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a232f9-31a8-4df3-9a13-893e7c04a2b1 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:30.390849 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 252/514] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:21:30.392126 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a232f9-31a8-4df3-9a13-893e7c04a2b1 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:30.396800 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87d84ab-0e45-4261-ba8c-db5c8b46a501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:30.397535 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d84ab-0e45-4261-ba8c-db5c8b46a501 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:30.397814 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d84ab-0e45-4261-ba8c-db5c8b46a501 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:30.398087 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d84ab-0e45-4261-ba8c-db5c8b46a501 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:30.398674 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87d84ab-0e45-4261-ba8c-db5c8b46a501 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:30.400077 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87d84ab-0e45-4261-ba8c-db5c8b46a501 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:30.410999 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-d87d84ab-0e45-4261-ba8c-db5c8b46a501 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 03:21:30.410999 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:30.410999 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 03:21:30.410999 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 03:21:30.410999 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 03:21:30.410999 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 03:21:30.410999 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 03:21:30.410999 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 03:21:30.410999 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 03:21:30.410999 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:30.410999 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 03:21:30.410999 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:30.410999 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:30.410999 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:30.410999 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:30.410999 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:30.410999 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:30.412047 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:30.412047 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:30.412047 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:30.412047 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:30.412047 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:30.412047 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:30.412047 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 03:21:30.412047 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 03:21:30.412047 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 03:21:30.412047 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 03:21:30.412047 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:30.412047 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:30.412047 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:30.412047 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:30.412047 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:30.412047 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:30.413114 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:30.413114 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:30.413114 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:30.413114 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:30.413114 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:30.413114 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:30.413114 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:30.413114 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:30.413114 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:30.413114 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:30.413114 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 03:21:30.413114 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 03:21:30.413114 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 03:21:30.413114 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:30.420422 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 263/515] 10.222.0.26 () {56 vars in 1129 bytes} [Wed Nov 2 03:21:30 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 03:21:30.420422 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea6016a-cc27-402d-97b1-f19d4668e411 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:30.420422 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea6016a-cc27-402d-97b1-f19d4668e411 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:30.420422 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea6016a-cc27-402d-97b1-f19d4668e411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:30.440515 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-bea6016a-cc27-402d-97b1-f19d4668e411 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:30.442222 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-bea6016a-cc27-402d-97b1-f19d4668e411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:30.504574 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a232f9-31a8-4df3-9a13-893e7c04a2b1 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:30.514202 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 264/516] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 03:21:30.520797 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea6016a-cc27-402d-97b1-f19d4668e411 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:30.552882 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1956ea56-5b0a-4bb2-92f6-edf99e07e6e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:30.553527 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1956ea56-5b0a-4bb2-92f6-edf99e07e6e1 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:30.553861 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1956ea56-5b0a-4bb2-92f6-edf99e07e6e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:30.554133 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1956ea56-5b0a-4bb2-92f6-edf99e07e6e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:30.556245 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:30.566003 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1956ea56-5b0a-4bb2-92f6-edf99e07e6e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:30.567382 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1956ea56-5b0a-4bb2-92f6-edf99e07e6e1 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:30.583440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-1956ea56-5b0a-4bb2-92f6-edf99e07e6e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:30.583440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:30.583440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:30.583440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:30.583440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:30.583440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:30.583440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:30.583440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:30.583440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:30.583440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:30.583440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:30.583440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:30.583440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 03:21:30.583440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 03:21:30.583440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 03:21:30.583440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 03:21:30.584460 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:30.584460 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:30.584460 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:30.584460 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:30.584460 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:30.584460 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:30.584460 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:30.584460 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:30.584460 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:30.584460 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:30.584460 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:30.584460 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:30.584460 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:30.584460 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:30.584460 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:30.584460 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:30.584460 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 03:21:30.585937 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 03:21:30.585937 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:30.585937 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:30.589793 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 253/517] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:21:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:30.590683 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1956ea56-5b0a-4bb2-92f6-edf99e07e6e1 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:30.644157 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:30.654566 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea6016a-cc27-402d-97b1-f19d4668e411 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:30.690968 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a631273c-96b0-4b9c-aafc-86a0084dbe7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:30.691594 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a631273c-96b0-4b9c-aafc-86a0084dbe7b None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:30.691878 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a631273c-96b0-4b9c-aafc-86a0084dbe7b None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:30.692192 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a631273c-96b0-4b9c-aafc-86a0084dbe7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:30.703429 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a631273c-96b0-4b9c-aafc-86a0084dbe7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:30.704905 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a631273c-96b0-4b9c-aafc-86a0084dbe7b None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:30.709743 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1956ea56-5b0a-4bb2-92f6-edf99e07e6e1 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:30.720799 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-a631273c-96b0-4b9c-aafc-86a0084dbe7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:30.720799 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:30.720799 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:30.720799 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:30.720799 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:30.720799 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:30.720799 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:30.720799 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:30.720799 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:30.720799 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:30.720799 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:30.720799 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:30.720799 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 03:21:30.720799 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 03:21:30.720799 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 03:21:30.720799 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 03:21:30.721777 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:30.721777 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:30.721777 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:30.721777 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:30.721777 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:30.721777 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:30.721777 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:30.721777 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:30.721777 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:30.721777 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:30.721777 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:30.721777 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:30.721777 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:30.721777 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:30.721777 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:30.721777 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:30.721777 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 03:21:30.722667 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 03:21:30.722667 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:30.722667 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:30.722667 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 265/518] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:21:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:30.722667 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a631273c-96b0-4b9c-aafc-86a0084dbe7b None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:30.753790 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66563353-1066-413b-89a5-3a4725095511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:30.754408 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66563353-1066-413b-89a5-3a4725095511 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:30.754688 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66563353-1066-413b-89a5-3a4725095511 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:30.754960 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66563353-1066-413b-89a5-3a4725095511 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:30.755652 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66563353-1066-413b-89a5-3a4725095511 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:30.755886 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66563353-1066-413b-89a5-3a4725095511 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:30.757845 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66563353-1066-413b-89a5-3a4725095511 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:30.763470 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 254/519] 10.222.0.26 () {56 vars in 1124 bytes} [Wed Nov 2 03:21:30 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:30.768396 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66563353-1066-413b-89a5-3a4725095511 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:30.822526 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:30.835563 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a631273c-96b0-4b9c-aafc-86a0084dbe7b None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:30.871874 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e7cf86-2ac0-4d25-bd8c-d2c3ba5999df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:30.872515 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e7cf86-2ac0-4d25-bd8c-d2c3ba5999df None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:30.872792 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e7cf86-2ac0-4d25-bd8c-d2c3ba5999df None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:30.873067 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e7cf86-2ac0-4d25-bd8c-d2c3ba5999df None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:30.873706 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e7cf86-2ac0-4d25-bd8c-d2c3ba5999df None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:30.874009 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e7cf86-2ac0-4d25-bd8c-d2c3ba5999df None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:30.875372 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e7cf86-2ac0-4d25-bd8c-d2c3ba5999df None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:30.877091 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66563353-1066-413b-89a5-3a4725095511 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:30.888183 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 266/520] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Nov 2 03:21:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:30.889517 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e7cf86-2ac0-4d25-bd8c-d2c3ba5999df None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:30.895082 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:30.912387 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2a90e9-2dd1-419b-a007-b4504d23b53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:30.913064 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a90e9-2dd1-419b-a007-b4504d23b53a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:30.913346 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a90e9-2dd1-419b-a007-b4504d23b53a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:30.913617 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2a90e9-2dd1-419b-a007-b4504d23b53a None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:30.914252 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2a90e9-2dd1-419b-a007-b4504d23b53a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:30.914551 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2a90e9-2dd1-419b-a007-b4504d23b53a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:30.915914 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2a90e9-2dd1-419b-a007-b4504d23b53a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:30.928378 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 255/521] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Nov 2 03:21:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:30.929571 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2a90e9-2dd1-419b-a007-b4504d23b53a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:30.999143 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e7cf86-2ac0-4d25-bd8c-d2c3ba5999df None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:31.036979 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515e53c0-4722-49e5-9415-03f541a59bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:31.037712 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515e53c0-4722-49e5-9415-03f541a59bd7 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:31.038042 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515e53c0-4722-49e5-9415-03f541a59bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:31.038438 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515e53c0-4722-49e5-9415-03f541a59bd7 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:31.039087 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515e53c0-4722-49e5-9415-03f541a59bd7 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:31.039369 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515e53c0-4722-49e5-9415-03f541a59bd7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:31.040763 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515e53c0-4722-49e5-9415-03f541a59bd7 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:31.041155 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2a90e9-2dd1-419b-a007-b4504d23b53a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:31.056108 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 267/522] 10.222.0.26 () {56 vars in 1124 bytes} [Wed Nov 2 03:21:30 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:31.056108 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515e53c0-4722-49e5-9415-03f541a59bd7 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:31.060936 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:31.089750 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1695d1d8-3eca-4e65-b61c-e57435cd5bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:31.090294 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1695d1d8-3eca-4e65-b61c-e57435cd5bdd None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:31.090561 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1695d1d8-3eca-4e65-b61c-e57435cd5bdd None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:31.090829 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1695d1d8-3eca-4e65-b61c-e57435cd5bdd None admin] PATH_INFO: `/v3/services/network` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:31.091424 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1695d1d8-3eca-4e65-b61c-e57435cd5bdd None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:31.092886 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1695d1d8-3eca-4e65-b61c-e57435cd5bdd None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:31.105888 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-1695d1d8-3eca-4e65-b61c-e57435cd5bdd None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 03:21:31.105888 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:31.105888 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 03:21:31.105888 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 03:21:31.105888 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 03:21:31.105888 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 03:21:31.105888 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 03:21:31.105888 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 03:21:31.105888 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 03:21:31.105888 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:31.105888 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 03:21:31.105888 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:31.105888 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:31.105888 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:31.105888 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:31.105888 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:31.105888 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:31.106951 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:31.106951 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:31.106951 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:31.106951 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:31.106951 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:31.106951 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:31.106951 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 03:21:31.106951 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 03:21:31.106951 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 03:21:31.106951 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 03:21:31.106951 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:31.106951 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:31.106951 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:31.106951 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:31.106951 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:31.106951 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:31.107838 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:31.107838 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:31.107838 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:31.107838 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:31.107838 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:31.107838 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:31.107838 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:31.107838 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:31.107838 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:31.107838 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:31.107838 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 03:21:31.107838 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 03:21:31.107838 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 03:21:31.107838 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:31.107838 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 256/523] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Nov 2 03:21:30 2022] GET /identity/v3/services/network => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:31.113038 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1695d1d8-3eca-4e65-b61c-e57435cd5bdd None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:31.143696 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:31.188077 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515e53c0-4722-49e5-9415-03f541a59bd7 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:31.227273 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2212ea2-7e78-47ad-b092-bfbd93a46689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:31.227923 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2212ea2-7e78-47ad-b092-bfbd93a46689 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:31.228278 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2212ea2-7e78-47ad-b092-bfbd93a46689 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:31.228553 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2212ea2-7e78-47ad-b092-bfbd93a46689 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:31.236201 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2212ea2-7e78-47ad-b092-bfbd93a46689 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:31.237920 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2212ea2-7e78-47ad-b092-bfbd93a46689 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:31.243241 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1695d1d8-3eca-4e65-b61c-e57435cd5bdd None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:31.252505 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-e2212ea2-7e78-47ad-b092-bfbd93a46689 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 03:21:31.252505 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:31.252505 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:31.252505 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:31.252505 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:31.252505 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:31.252505 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:31.252505 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:31.252505 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:31.252505 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:31.252505 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:31.252505 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:31.252505 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 03:21:31.252505 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 03:21:31.252505 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 03:21:31.253488 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 03:21:31.253488 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:31.253488 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:31.253488 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:31.253488 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:31.253488 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:31.253488 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:31.253488 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:31.253488 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:31.253488 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:31.253488 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:31.253488 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:31.253488 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:31.253488 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:31.253488 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:31.253488 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:31.253488 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:31.254395 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 03:21:31.254395 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 03:21:31.254395 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 03:21:31.254395 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 03:21:31.254395 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 03:21:31.254395 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 03:21:31.254395 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 03:21:31.254395 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:31.259142 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 268/524] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Nov 2 03:21:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:31.259818 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2212ea2-7e78-47ad-b092-bfbd93a46689 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:31.294729 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b51cd4-eda6-4c89-b7ad-a598c402965f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:31.295605 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b51cd4-eda6-4c89-b7ad-a598c402965f None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:31.295929 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b51cd4-eda6-4c89-b7ad-a598c402965f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:31.296240 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b51cd4-eda6-4c89-b7ad-a598c402965f None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:31.297594 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b51cd4-eda6-4c89-b7ad-a598c402965f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:31.299147 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b51cd4-eda6-4c89-b7ad-a598c402965f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:31.324253 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 257/525] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Nov 2 03:21:31 2022] POST /identity/v3/users => generated 254 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:21:31.325794 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b51cd4-eda6-4c89-b7ad-a598c402965f None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:31.365078 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:31.413611 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2212ea2-7e78-47ad-b092-bfbd93a46689 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:31.453932 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b51cd4-eda6-4c89-b7ad-a598c402965f None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:31.466266 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:31.475233 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3bfb7f-f7c6-4865-94cf-8bf903295256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:31.476424 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3bfb7f-f7c6-4865-94cf-8bf903295256 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:31.476802 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3bfb7f-f7c6-4865-94cf-8bf903295256 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:31.477081 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3bfb7f-f7c6-4865-94cf-8bf903295256 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:31.477820 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3bfb7f-f7c6-4865-94cf-8bf903295256 None admin] RBAC: Adding query filter params (name=network) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:31.478181 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3bfb7f-f7c6-4865-94cf-8bf903295256 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:31.479692 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3bfb7f-f7c6-4865-94cf-8bf903295256 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:31.493617 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 269/526] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Nov 2 03:21:31 2022] GET /identity/v3/services?name=network => generated 124 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:31.499140 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3bfb7f-f7c6-4865-94cf-8bf903295256 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:31.501859 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbe85f0-5347-4e86-ba2d-94dd1bb416fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:31.502504 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe85f0-5347-4e86-ba2d-94dd1bb416fd None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:31.502779 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe85f0-5347-4e86-ba2d-94dd1bb416fd None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:31.503047 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe85f0-5347-4e86-ba2d-94dd1bb416fd None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:31.503750 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbe85f0-5347-4e86-ba2d-94dd1bb416fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:31.504242 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbe85f0-5347-4e86-ba2d-94dd1bb416fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:31.505703 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbe85f0-5347-4e86-ba2d-94dd1bb416fd None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:31.520699 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 258/527] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Nov 2 03:21:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:31.527066 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbe85f0-5347-4e86-ba2d-94dd1bb416fd None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:31.634474 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:31.646773 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbe85f0-5347-4e86-ba2d-94dd1bb416fd None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:31.677526 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3bfb7f-f7c6-4865-94cf-8bf903295256 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:31.692652 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98db6995-c058-452a-9bf2-b5e3d9e8f538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:31.693323 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98db6995-c058-452a-9bf2-b5e3d9e8f538 None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:31.693661 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98db6995-c058-452a-9bf2-b5e3d9e8f538 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:31.693867 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98db6995-c058-452a-9bf2-b5e3d9e8f538 None admin] PATH_INFO: `/v3/projects/d0b5e7017ac64ae6ab62c1a05aa54b72/users/44961b06151c4da382fcd54dd1f1e146/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:31.696095 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98db6995-c058-452a-9bf2-b5e3d9e8f538 None admin] RBAC: Authorizing `identity:create_grant(project_id=d0b5e7017ac64ae6ab62c1a05aa54b72, user_id=44961b06151c4da382fcd54dd1f1e146, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:31.697536 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98db6995-c058-452a-9bf2-b5e3d9e8f538 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:31.721321 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 259/528] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Nov 2 03:21:31 2022] PUT /identity/v3/projects/d0b5e7017ac64ae6ab62c1a05aa54b72/users/44961b06151c4da382fcd54dd1f1e146/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 03:21:31.732807 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87056e4-ec1e-44b0-bfec-b7bc61b1911b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:31.733427 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87056e4-ec1e-44b0-bfec-b7bc61b1911b None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:31.733703 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87056e4-ec1e-44b0-bfec-b7bc61b1911b None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:31.733976 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87056e4-ec1e-44b0-bfec-b7bc61b1911b None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:31.734601 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87056e4-ec1e-44b0-bfec-b7bc61b1911b None admin] RBAC: Adding query filter params (type=network) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:31.734954 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87056e4-ec1e-44b0-bfec-b7bc61b1911b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:31.736337 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87056e4-ec1e-44b0-bfec-b7bc61b1911b None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:31.748816 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 270/529] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Nov 2 03:21:31 2022] GET /identity/v3/services?type=network => generated 124 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:32.411835 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c590fd-15fb-4654-a0b9-7e87868f07af None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:32.412244 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c590fd-15fb-4654-a0b9-7e87868f07af None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:32.412419 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c590fd-15fb-4654-a0b9-7e87868f07af None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:32.414578 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 260/530] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:21:32.419823 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de2c4ad-1443-4e2b-bd94-3c33ee50ef50 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:32.420139 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de2c4ad-1443-4e2b-bd94-3c33ee50ef50 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:32.420412 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de2c4ad-1443-4e2b-bd94-3c33ee50ef50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:32.446030 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-5de2c4ad-1443-4e2b-bd94-3c33ee50ef50 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:32.447233 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-5de2c4ad-1443-4e2b-bd94-3c33ee50ef50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:32.520291 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:32.539981 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 271/531] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 03:21:33.009407 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c590fd-15fb-4654-a0b9-7e87868f07af None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:33.116551 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c590fd-15fb-4654-a0b9-7e87868f07af None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:33.152565 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499e5aba-5d73-406d-ba1a-14cf9554d139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:33.153202 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499e5aba-5d73-406d-ba1a-14cf9554d139 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:33.153478 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499e5aba-5d73-406d-ba1a-14cf9554d139 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:33.153748 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499e5aba-5d73-406d-ba1a-14cf9554d139 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:33.164865 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499e5aba-5d73-406d-ba1a-14cf9554d139 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:21:33.169935 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-499e5aba-5d73-406d-ba1a-14cf9554d139 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 03:21:33.169935 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:33.169935 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:33.169935 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:33.169935 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:33.169935 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:33.169935 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:33.169935 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:33.169935 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:33.169935 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:33.169935 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:33.169935 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:33.169935 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:21:33.169935 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:21:33.169935 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:21:33.169935 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:21:33.170857 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:21:33.170857 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:21:33.170857 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:21:33.170857 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:21:33.170857 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:33.170857 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:33.170857 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:33.170857 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:33.170857 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:33.170857 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:33.170857 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:33.170857 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:33.170857 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:33.170857 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:33.170857 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:33.170857 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:33.171672 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:33.171672 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:33.171672 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:33.171672 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:33.171672 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:21:33.171672 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:21:33.171672 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:21:33.171672 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:21:33.171672 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:21:33.171672 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:21:33.171672 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 03:21:33.171672 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:33.175938 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 261/532] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Nov 2 03:21:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:33.177316 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de2c4ad-1443-4e2b-bd94-3c33ee50ef50 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:33.281854 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de2c4ad-1443-4e2b-bd94-3c33ee50ef50 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:33.321601 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb56c285-70ce-4734-8afe-95c9d51d1910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:33.322586 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb56c285-70ce-4734-8afe-95c9d51d1910 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:33.323087 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb56c285-70ce-4734-8afe-95c9d51d1910 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:33.323552 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb56c285-70ce-4734-8afe-95c9d51d1910 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:33.324497 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb56c285-70ce-4734-8afe-95c9d51d1910 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:33.325012 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb56c285-70ce-4734-8afe-95c9d51d1910 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:33.326566 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb56c285-70ce-4734-8afe-95c9d51d1910 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:33.339770 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 272/533] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Nov 2 03:21:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:33.345519 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499e5aba-5d73-406d-ba1a-14cf9554d139 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:33.367525 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:33.479484 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499e5aba-5d73-406d-ba1a-14cf9554d139 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:33.514719 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8067b4-aeee-4a41-a494-26339bab4057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:33.515629 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8067b4-aeee-4a41-a494-26339bab4057 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:33.516086 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8067b4-aeee-4a41-a494-26339bab4057 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:33.516575 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8067b4-aeee-4a41-a494-26339bab4057 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:33.527114 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8067b4-aeee-4a41-a494-26339bab4057 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:33.528781 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8067b4-aeee-4a41-a494-26339bab4057 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:33.552342 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-9f8067b4-aeee-4a41-a494-26339bab4057 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:33.552342 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:33.552342 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:33.552342 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:33.552342 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:33.552342 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:33.552342 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:33.552342 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:33.552342 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:33.552342 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:33.552342 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:33.552342 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:33.552342 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 03:21:33.552342 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 03:21:33.552342 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 03:21:33.552342 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 03:21:33.553486 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:33.553486 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:33.553486 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:33.553486 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:33.553486 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:33.553486 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:33.553486 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:33.553486 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:33.553486 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:33.553486 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:33.553486 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:33.553486 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:33.553486 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:33.553486 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:33.553486 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:33.553486 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:33.553486 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 03:21:33.554645 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 03:21:33.554645 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:33.554645 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:33.560050 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 262/534] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:21:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:33.562237 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb56c285-70ce-4734-8afe-95c9d51d1910 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:33.614313 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:33.669364 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb56c285-70ce-4734-8afe-95c9d51d1910 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:33.705220 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307bf4f0-f923-4841-af0e-e9757c69b7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:33.705853 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307bf4f0-f923-4841-af0e-e9757c69b7d7 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:33.706134 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307bf4f0-f923-4841-af0e-e9757c69b7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:33.706410 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307bf4f0-f923-4841-af0e-e9757c69b7d7 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:33.707390 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307bf4f0-f923-4841-af0e-e9757c69b7d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:33.707698 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307bf4f0-f923-4841-af0e-e9757c69b7d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:33.709177 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307bf4f0-f923-4841-af0e-e9757c69b7d7 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:33.728103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 273/535] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Nov 2 03:21:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:33.730033 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8067b4-aeee-4a41-a494-26339bab4057 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:33.800335 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:33.828721 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8377760c-35e0-4d0f-9110-1f95ff4e1d58 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:33.828909 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8377760c-35e0-4d0f-9110-1f95ff4e1d58 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:33.829189 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8377760c-35e0-4d0f-9110-1f95ff4e1d58 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:33.830526 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 274/536] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:21:33.839423 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8067b4-aeee-4a41-a494-26339bab4057 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:33.846050 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138aaf0-cbe4-454a-be30-be81a473a9b6 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:33.846590 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138aaf0-cbe4-454a-be30-be81a473a9b6 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:33.847089 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c138aaf0-cbe4-454a-be30-be81a473a9b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:33.877488 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbde82a6-46c7-48b4-bc6e-675565037d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:33.878530 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde82a6-46c7-48b4-bc6e-675565037d94 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:33.878974 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde82a6-46c7-48b4-bc6e-675565037d94 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:33.879463 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbde82a6-46c7-48b4-bc6e-675565037d94 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:33.884978 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-c138aaf0-cbe4-454a-be30-be81a473a9b6 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:33.886272 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-c138aaf0-cbe4-454a-be30-be81a473a9b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:33.887744 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbde82a6-46c7-48b4-bc6e-675565037d94 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:33.889568 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbde82a6-46c7-48b4-bc6e-675565037d94 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:33.902588 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-dbde82a6-46c7-48b4-bc6e-675565037d94 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Nov 02 03:21:33.902588 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:33.902588 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:33.902588 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:33.902588 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:33.902588 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:33.902588 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:33.902588 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:33.902588 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:33.902588 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:33.902588 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:33.902588 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:33.902588 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 03:21:33.902588 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 03:21:33.902588 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 03:21:33.902588 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 03:21:33.903747 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:33.903747 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:33.903747 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 03:21:33.903747 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 03:21:33.903747 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 03:21:33.903747 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 03:21:33.903747 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:33.903747 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:33.903747 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:33.903747 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:33.903747 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:33.903747 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:33.903747 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:33.903747 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:33.903747 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:33.903747 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:33.903747 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:33.903747 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:33.904832 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:33.904832 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:33.904832 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 03:21:33.904832 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 03:21:33.904832 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 03:21:33.904832 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 03:21:33.904832 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 03:21:33.904832 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 03:21:33.904832 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Nov 02 03:21:33.904832 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:33.912196 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 263/537] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Nov 2 03:21:33 2022] GET /identity/v3/users/swift => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:33.913614 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbde82a6-46c7-48b4-bc6e-675565037d94 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:34.025883 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbde82a6-46c7-48b4-bc6e-675565037d94 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:34.052497 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:34.064512 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260f7ac6-157b-4eb8-8777-b17d0dba5c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:34.065413 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f7ac6-157b-4eb8-8777-b17d0dba5c24 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:34.066084 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f7ac6-157b-4eb8-8777-b17d0dba5c24 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:34.066538 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260f7ac6-157b-4eb8-8777-b17d0dba5c24 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:34.067904 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260f7ac6-157b-4eb8-8777-b17d0dba5c24 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:34.068537 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260f7ac6-157b-4eb8-8777-b17d0dba5c24 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:34.070387 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-260f7ac6-157b-4eb8-8777-b17d0dba5c24 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:34.095216 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 264/538] 10.222.0.26 () {56 vars in 1135 bytes} [Wed Nov 2 03:21:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:21:34.096870 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260f7ac6-157b-4eb8-8777-b17d0dba5c24 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:34.146478 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 275/539] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 03:21:34.148385 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81578429-a20d-47ca-b43b-e27c71964930 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:34.148677 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81578429-a20d-47ca-b43b-e27c71964930 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:34.148957 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81578429-a20d-47ca-b43b-e27c71964930 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:34.150503 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 276/540] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:21:34.161232 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643545d0-eced-4189-8ebc-c0b5e2484c0e None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:34.161520 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643545d0-eced-4189-8ebc-c0b5e2484c0e None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:34.161800 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643545d0-eced-4189-8ebc-c0b5e2484c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:34.207046 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260f7ac6-157b-4eb8-8777-b17d0dba5c24 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:34.208080 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-643545d0-eced-4189-8ebc-c0b5e2484c0e None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:34.209412 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-643545d0-eced-4189-8ebc-c0b5e2484c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:34.252334 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:34.289008 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 277/541] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 03:21:34.332055 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0206366e-fbbf-4e8d-9cb6-c991b75ec2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:34.332716 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0206366e-fbbf-4e8d-9cb6-c991b75ec2f1 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:34.332996 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0206366e-fbbf-4e8d-9cb6-c991b75ec2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:34.333270 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0206366e-fbbf-4e8d-9cb6-c991b75ec2f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:34.358397 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0206366e-fbbf-4e8d-9cb6-c991b75ec2f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:34.359806 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0206366e-fbbf-4e8d-9cb6-c991b75ec2f1 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:34.385785 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-0206366e-fbbf-4e8d-9cb6-c991b75ec2f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:34.385785 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:34.385785 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:34.385785 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:34.385785 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:34.385785 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:34.385785 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:34.385785 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:34.385785 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:34.385785 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:34.385785 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:34.385785 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:34.385785 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 03:21:34.385785 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 03:21:34.385785 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 03:21:34.385785 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 03:21:34.386786 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:34.386786 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:34.386786 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:34.386786 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:34.386786 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:34.386786 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:34.386786 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:34.386786 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:34.386786 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:34.386786 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:34.386786 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:34.386786 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:34.386786 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:34.386786 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:34.386786 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:34.386786 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:34.386786 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 03:21:34.388057 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 03:21:34.388057 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:34.388057 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:34.388057 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 265/542] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:21:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:34.394094 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643545d0-eced-4189-8ebc-c0b5e2484c0e None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:34.463008 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df23d0f-829a-47f3-a1eb-8303a4c56d86 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:34.463339 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df23d0f-829a-47f3-a1eb-8303a4c56d86 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:34.463622 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df23d0f-829a-47f3-a1eb-8303a4c56d86 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:34.467091 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 266/543] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:21:34.477493 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e36d061-460f-424d-9375-8a276a3f708d None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:34.477809 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e36d061-460f-424d-9375-8a276a3f708d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:34.478044 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e36d061-460f-424d-9375-8a276a3f708d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:34.510019 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-3e36d061-460f-424d-9375-8a276a3f708d None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:34.511246 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3e36d061-460f-424d-9375-8a276a3f708d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:34.531373 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643545d0-eced-4189-8ebc-c0b5e2484c0e None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:34.544827 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:34.568497 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97723479-23c3-4a27-ad76-c2e3d8b685d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:34.569653 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97723479-23c3-4a27-ad76-c2e3d8b685d7 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:34.569949 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97723479-23c3-4a27-ad76-c2e3d8b685d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:34.570220 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97723479-23c3-4a27-ad76-c2e3d8b685d7 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:34.570908 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97723479-23c3-4a27-ad76-c2e3d8b685d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:34.571209 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97723479-23c3-4a27-ad76-c2e3d8b685d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:34.572616 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97723479-23c3-4a27-ad76-c2e3d8b685d7 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:34.585000 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 278/544] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Nov 2 03:21:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:34.585864 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97723479-23c3-4a27-ad76-c2e3d8b685d7 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:34.594494 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 267/545] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Nov 02 03:21:34.595520 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e36d061-460f-424d-9375-8a276a3f708d None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:34.630766 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:34.705700 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e36d061-460f-424d-9375-8a276a3f708d None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:34.706147 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97723479-23c3-4a27-ad76-c2e3d8b685d7 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:34.740561 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7604009-4b91-447f-9cda-178610955070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:34.741682 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7604009-4b91-447f-9cda-178610955070 None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:34.741962 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7604009-4b91-447f-9cda-178610955070 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:34.742234 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7604009-4b91-447f-9cda-178610955070 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:34.742876 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7604009-4b91-447f-9cda-178610955070 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:34.743147 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514c0f7b-53da-4c32-be13-91a28640d679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:34.744037 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514c0f7b-53da-4c32-be13-91a28640d679 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:34.744259 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7604009-4b91-447f-9cda-178610955070 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:34.744740 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514c0f7b-53da-4c32-be13-91a28640d679 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:34.745214 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514c0f7b-53da-4c32-be13-91a28640d679 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:34.753967 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514c0f7b-53da-4c32-be13-91a28640d679 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:34.756222 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514c0f7b-53da-4c32-be13-91a28640d679 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:34.760776 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 279/546] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Nov 2 03:21:34 2022] POST /identity/v3/services => generated 251 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:21:34.762126 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc95b60-cc42-42c4-9347-519bf7cb197f None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:34.762562 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc95b60-cc42-42c4-9347-519bf7cb197f None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:34.762833 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc95b60-cc42-42c4-9347-519bf7cb197f None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:34.764340 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 280/547] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:21:34.765078 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc95b60-cc42-42c4-9347-519bf7cb197f None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:34.777937 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-514c0f7b-53da-4c32-be13-91a28640d679 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 03:21:34.777937 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:34.777937 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:34.777937 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:34.777937 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:34.777937 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:34.777937 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:34.777937 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:34.777937 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:34.777937 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:34.777937 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:34.777937 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:34.777937 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 03:21:34.777937 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 03:21:34.777937 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 03:21:34.778959 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 03:21:34.778959 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:34.778959 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:34.778959 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:34.778959 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:34.778959 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:34.778959 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:34.778959 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:34.778959 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:34.778959 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:34.778959 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:34.778959 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:34.778959 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:34.778959 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:34.778959 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:34.778959 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:34.778959 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:34.779890 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 03:21:34.779890 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 03:21:34.779890 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 03:21:34.779890 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 03:21:34.779890 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 03:21:34.779890 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 03:21:34.779890 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 03:21:34.779890 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:34.782543 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 268/548] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Nov 2 03:21:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:34.784404 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0740ac3-d2a8-4010-8a65-d6d635206d8c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:34.785016 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0740ac3-d2a8-4010-8a65-d6d635206d8c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:34.785519 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0740ac3-d2a8-4010-8a65-d6d635206d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:34.806574 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-c0740ac3-d2a8-4010-8a65-d6d635206d8c None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:34.808170 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c0740ac3-d2a8-4010-8a65-d6d635206d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:34.827360 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:34.884370 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:34.906503 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 269/549] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:34 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Nov 02 03:21:34.919979 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0740ac3-d2a8-4010-8a65-d6d635206d8c None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:34.920185 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc95b60-cc42-42c4-9347-519bf7cb197f None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:34.957326 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ab2056-74db-434c-901a-fe264b648837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:34.958541 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ab2056-74db-434c-901a-fe264b648837 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:34.958838 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ab2056-74db-434c-901a-fe264b648837 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:34.959116 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ab2056-74db-434c-901a-fe264b648837 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:34.972164 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ab2056-74db-434c-901a-fe264b648837 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:21:34.978131 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-a8ab2056-74db-434c-901a-fe264b648837 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 03:21:34.978131 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:34.978131 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:34.978131 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:34.978131 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:34.978131 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:34.978131 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:34.978131 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:34.978131 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:34.978131 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:34.978131 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:34.978131 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:34.978131 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:21:34.978131 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:21:34.978131 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:21:34.978131 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:21:34.979042 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:21:34.979042 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:21:34.979042 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:21:34.979042 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:21:34.979042 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:34.979042 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:34.979042 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:34.979042 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:34.979042 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:34.979042 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:34.979042 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:34.979042 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:34.979042 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:34.979042 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:34.979042 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:34.979042 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:34.979935 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:34.979935 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:34.979935 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:34.979935 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:34.979935 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:21:34.979935 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:21:34.979935 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:21:34.979935 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:21:34.979935 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:21:34.979935 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:21:34.979935 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 03:21:34.979935 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:34.979935 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 281/550] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Nov 2 03:21:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:34.983503 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ab2056-74db-434c-901a-fe264b648837 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:35.034812 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0740ac3-d2a8-4010-8a65-d6d635206d8c None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:35.076731 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b0cb9-f706-4393-b432-6fae3e28a59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:35.077360 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b0cb9-f706-4393-b432-6fae3e28a59e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:35.077642 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b0cb9-f706-4393-b432-6fae3e28a59e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:35.078030 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b0cb9-f706-4393-b432-6fae3e28a59e None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:35.078848 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87b0cb9-f706-4393-b432-6fae3e28a59e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:35.079158 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87b0cb9-f706-4393-b432-6fae3e28a59e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:35.080641 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87b0cb9-f706-4393-b432-6fae3e28a59e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:35.097030 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 270/551] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Nov 2 03:21:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:35.097884 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b0cb9-f706-4393-b432-6fae3e28a59e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:35.098670 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ab2056-74db-434c-901a-fe264b648837 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:35.110634 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:35.141310 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3761ae16-1e4a-4af4-b9bd-249f6fc85f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:35.142269 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3761ae16-1e4a-4af4-b9bd-249f6fc85f7b None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:35.142695 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3761ae16-1e4a-4af4-b9bd-249f6fc85f7b None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:35.143114 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3761ae16-1e4a-4af4-b9bd-249f6fc85f7b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:35.154917 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3761ae16-1e4a-4af4-b9bd-249f6fc85f7b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:21:35.160995 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-3761ae16-1e4a-4af4-b9bd-249f6fc85f7b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 03:21:35.160995 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:35.160995 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:35.160995 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:35.160995 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:35.160995 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:35.160995 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:35.160995 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:35.160995 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:35.160995 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:35.160995 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:35.160995 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:35.160995 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:21:35.160995 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:21:35.160995 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:21:35.160995 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:21:35.161984 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:21:35.161984 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:21:35.161984 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:21:35.161984 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:21:35.161984 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:35.161984 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:35.161984 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:35.161984 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:35.161984 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:35.161984 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:35.161984 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:35.161984 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:35.161984 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:35.161984 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:35.161984 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:35.161984 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:35.162856 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:35.162856 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:35.162856 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:35.162856 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:35.162856 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:21:35.162856 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:21:35.162856 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:21:35.162856 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:21:35.162856 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:21:35.162856 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:21:35.162856 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 03:21:35.162856 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:35.165663 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 282/552] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Nov 2 03:21:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:35.166599 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3761ae16-1e4a-4af4-b9bd-249f6fc85f7b None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:35.177084 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:35.229359 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b0cb9-f706-4393-b432-6fae3e28a59e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:35.268767 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cef6b5b-ab8e-4c8c-b39b-d50f3070699b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:35.269373 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef6b5b-ab8e-4c8c-b39b-d50f3070699b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:35.269652 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef6b5b-ab8e-4c8c-b39b-d50f3070699b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:35.269925 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef6b5b-ab8e-4c8c-b39b-d50f3070699b None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:35.270586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cef6b5b-ab8e-4c8c-b39b-d50f3070699b None admin] RBAC: Adding query filter params (name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:35.270888 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cef6b5b-ab8e-4c8c-b39b-d50f3070699b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:35.272829 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cef6b5b-ab8e-4c8c-b39b-d50f3070699b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:35.280653 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 271/553] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Nov 2 03:21:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:35.281425 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cef6b5b-ab8e-4c8c-b39b-d50f3070699b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:35.290226 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3761ae16-1e4a-4af4-b9bd-249f6fc85f7b None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:35.327866 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ec026-320f-41d3-8c77-1579733907cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:35.328775 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ec026-320f-41d3-8c77-1579733907cd None admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:35.329218 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ec026-320f-41d3-8c77-1579733907cd None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:35.329687 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ec026-320f-41d3-8c77-1579733907cd None admin] PATH_INFO: `/v3/projects/d0b5e7017ac64ae6ab62c1a05aa54b72/users/5193c40a917b4105ab8b27aead1685f8/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:35.331891 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0ec026-320f-41d3-8c77-1579733907cd None admin] RBAC: Authorizing `identity:create_grant(project_id=d0b5e7017ac64ae6ab62c1a05aa54b72, user_id=5193c40a917b4105ab8b27aead1685f8, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:35.333724 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0ec026-320f-41d3-8c77-1579733907cd None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:35.348162 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 283/554] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Nov 2 03:21:35 2022] PUT /identity/v3/projects/d0b5e7017ac64ae6ab62c1a05aa54b72/users/5193c40a917b4105ab8b27aead1685f8/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:21:35.349213 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ec026-320f-41d3-8c77-1579733907cd None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:35.361565 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:35.405961 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cef6b5b-ab8e-4c8c-b39b-d50f3070699b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:35.446978 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddef243-d7c0-4b9c-b387-6bc9f2b2fb36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:35.447603 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddef243-d7c0-4b9c-b387-6bc9f2b2fb36 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:35.447879 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddef243-d7c0-4b9c-b387-6bc9f2b2fb36 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:35.448238 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ddef243-d7c0-4b9c-b387-6bc9f2b2fb36 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:35.449036 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddef243-d7c0-4b9c-b387-6bc9f2b2fb36 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:35.449343 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddef243-d7c0-4b9c-b387-6bc9f2b2fb36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:35.451024 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ddef243-d7c0-4b9c-b387-6bc9f2b2fb36 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:35.465551 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 272/555] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Nov 2 03:21:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:35.465551 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddef243-d7c0-4b9c-b387-6bc9f2b2fb36 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:35.486960 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:35.544933 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ec026-320f-41d3-8c77-1579733907cd None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:35.587847 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ddef243-d7c0-4b9c-b387-6bc9f2b2fb36 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:35.589722 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ee4e0-555f-416e-a9e1-8b4763335f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:35.590332 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ee4e0-555f-416e-a9e1-8b4763335f5b None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:35.590610 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ee4e0-555f-416e-a9e1-8b4763335f5b None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:35.590882 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001ee4e0-555f-416e-a9e1-8b4763335f5b None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:35.591536 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001ee4e0-555f-416e-a9e1-8b4763335f5b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:35.592957 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001ee4e0-555f-416e-a9e1-8b4763335f5b None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:35.616460 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 284/556] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Nov 2 03:21:35 2022] POST /identity/v3/services => generated 240 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:21:35.616460 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ee4e0-555f-416e-a9e1-8b4763335f5b None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:35.642132 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:35.680925 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee7f5a8-fbcf-41d8-b94b-c3d27bf79216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:35.681504 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7f5a8-fbcf-41d8-b94b-c3d27bf79216 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:35.681785 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7f5a8-fbcf-41d8-b94b-c3d27bf79216 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:35.682061 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7f5a8-fbcf-41d8-b94b-c3d27bf79216 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:35.699499 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee7f5a8-fbcf-41d8-b94b-c3d27bf79216 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:35.709239 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee7f5a8-fbcf-41d8-b94b-c3d27bf79216 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:35.726025 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-1ee7f5a8-fbcf-41d8-b94b-c3d27bf79216 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:35.726025 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:35.726025 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:35.726025 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:35.726025 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:35.726025 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:35.726025 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:35.726025 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:35.726025 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:35.726025 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:35.726025 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:35.726025 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:35.726025 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 03:21:35.726025 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 03:21:35.726025 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 03:21:35.726025 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 03:21:35.727023 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:35.727023 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:35.727023 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:35.727023 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:35.727023 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:35.727023 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:35.727023 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:35.727023 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:35.727023 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:35.727023 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:35.727023 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:35.727023 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:35.727023 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:35.727023 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:35.727023 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:35.727023 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:35.727023 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 03:21:35.727912 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 03:21:35.727912 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:35.727912 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:35.727912 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 273/557] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:21:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:35.734568 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee7f5a8-fbcf-41d8-b94b-c3d27bf79216 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:35.778966 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001ee4e0-555f-416e-a9e1-8b4763335f5b None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:35.828101 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:35.844188 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b30ddd8-65e5-495e-b489-10c1095bffa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:35.846048 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30ddd8-65e5-495e-b489-10c1095bffa2 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:35.846333 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30ddd8-65e5-495e-b489-10c1095bffa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:35.846611 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b30ddd8-65e5-495e-b489-10c1095bffa2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:35.862206 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b30ddd8-65e5-495e-b489-10c1095bffa2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:35.864805 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b30ddd8-65e5-495e-b489-10c1095bffa2 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:35.883382 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee7f5a8-fbcf-41d8-b94b-c3d27bf79216 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:35.885519 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-7b30ddd8-65e5-495e-b489-10c1095bffa2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:35.885519 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:35.885519 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:35.885519 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:35.885519 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:35.885519 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:35.885519 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:35.885519 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:35.885519 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:35.885519 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:35.885519 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:35.885519 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:35.885519 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 03:21:35.885519 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 03:21:35.885519 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 03:21:35.885519 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 03:21:35.886431 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:35.886431 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:35.886431 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:35.886431 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:35.886431 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:35.886431 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:35.886431 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:35.886431 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:35.886431 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:35.886431 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:35.886431 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:35.886431 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:35.886431 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:35.886431 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:35.886431 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:35.886431 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:35.886431 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 03:21:35.887307 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 03:21:35.887307 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:35.887307 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:35.888621 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 285/558] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:21:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:35.893585 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b30ddd8-65e5-495e-b489-10c1095bffa2 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:35.927873 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d8aaf2-4458-4c25-97b6-6abdff3dc3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:35.928725 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d8aaf2-4458-4c25-97b6-6abdff3dc3f4 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:35.928904 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d8aaf2-4458-4c25-97b6-6abdff3dc3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:35.929217 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d8aaf2-4458-4c25-97b6-6abdff3dc3f4 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:35.929950 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d8aaf2-4458-4c25-97b6-6abdff3dc3f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:35.930154 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d8aaf2-4458-4c25-97b6-6abdff3dc3f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:35.931590 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d8aaf2-4458-4c25-97b6-6abdff3dc3f4 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:35.945577 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 274/559] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Nov 2 03:21:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:35.950768 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d8aaf2-4458-4c25-97b6-6abdff3dc3f4 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:35.998118 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:36.026674 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b30ddd8-65e5-495e-b489-10c1095bffa2 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:36.081231 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d8aaf2-4458-4c25-97b6-6abdff3dc3f4 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:36.093413 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eb9f22-0a8a-4088-9dd7-666a27db30e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:36.094034 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb9f22-0a8a-4088-9dd7-666a27db30e1 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:36.094312 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb9f22-0a8a-4088-9dd7-666a27db30e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:36.094586 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb9f22-0a8a-4088-9dd7-666a27db30e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:36.095237 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eb9f22-0a8a-4088-9dd7-666a27db30e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:36.095540 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eb9f22-0a8a-4088-9dd7-666a27db30e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:36.096951 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eb9f22-0a8a-4088-9dd7-666a27db30e1 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:36.121117 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 286/560] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Nov 2 03:21:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:21:36.128030 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eb9f22-0a8a-4088-9dd7-666a27db30e1 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:36.149364 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092fec6f-008e-4b8f-80cb-0a2759c0e389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:36.150258 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092fec6f-008e-4b8f-80cb-0a2759c0e389 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:36.150545 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092fec6f-008e-4b8f-80cb-0a2759c0e389 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:36.150821 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092fec6f-008e-4b8f-80cb-0a2759c0e389 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:36.165504 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092fec6f-008e-4b8f-80cb-0a2759c0e389 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:36.167256 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-092fec6f-008e-4b8f-80cb-0a2759c0e389 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:36.190252 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-092fec6f-008e-4b8f-80cb-0a2759c0e389 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 03:21:36.190252 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:36.190252 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:36.190252 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:36.190252 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:36.190252 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:36.190252 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:36.190252 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:36.190252 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:36.190252 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:36.190252 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:36.190252 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:36.190252 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 03:21:36.190252 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 03:21:36.190252 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 03:21:36.190252 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 03:21:36.191215 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:36.191215 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:36.191215 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 03:21:36.191215 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 03:21:36.191215 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 03:21:36.191215 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 03:21:36.191215 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:36.191215 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:36.191215 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:36.191215 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:36.191215 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:36.191215 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:36.191215 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:36.191215 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:36.191215 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:36.191215 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:36.191215 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:36.191215 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:36.192185 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:36.192185 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:36.192185 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 03:21:36.192185 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 03:21:36.192185 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 03:21:36.192185 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 03:21:36.192185 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 03:21:36.192185 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 03:21:36.192185 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 03:21:36.192185 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:36.192726 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 275/561] 10.222.0.26 () {56 vars in 1117 bytes} [Wed Nov 2 03:21:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:36.196384 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:36.197090 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092fec6f-008e-4b8f-80cb-0a2759c0e389 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:36.278740 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eb9f22-0a8a-4088-9dd7-666a27db30e1 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:36.314644 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdd3033-7772-4e28-a7f0-067cc02f5327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:36.315274 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd3033-7772-4e28-a7f0-067cc02f5327 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:36.315532 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd3033-7772-4e28-a7f0-067cc02f5327 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:36.315799 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdd3033-7772-4e28-a7f0-067cc02f5327 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:36.323550 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdd3033-7772-4e28-a7f0-067cc02f5327 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:36.325140 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdd3033-7772-4e28-a7f0-067cc02f5327 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:36.337862 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-fcdd3033-7772-4e28-a7f0-067cc02f5327 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Nov 02 03:21:36.337862 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:36.337862 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:36.337862 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:36.337862 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:36.337862 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:36.337862 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:36.337862 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:36.337862 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:36.337862 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:36.337862 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:36.337862 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:36.337862 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 03:21:36.337862 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 03:21:36.337862 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 03:21:36.337862 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 03:21:36.338846 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:36.338846 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:36.338846 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 03:21:36.338846 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 03:21:36.338846 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 03:21:36.338846 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 03:21:36.338846 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:36.338846 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:36.338846 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:36.338846 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:36.338846 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:36.338846 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:36.338846 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:36.338846 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:36.338846 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:36.338846 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:36.338846 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:36.338846 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:36.339778 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:36.339778 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:36.339778 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 03:21:36.339778 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 03:21:36.339778 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 03:21:36.339778 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 03:21:36.339778 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 03:21:36.339778 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 03:21:36.339778 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Nov 02 03:21:36.339778 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:36.339778 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 287/562] 10.222.0.26 () {56 vars in 1093 bytes} [Wed Nov 2 03:21:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 03:21:36.344822 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdd3033-7772-4e28-a7f0-067cc02f5327 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:36.461841 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:36.466155 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092fec6f-008e-4b8f-80cb-0a2759c0e389 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:36.488930 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:36.494693 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdd3033-7772-4e28-a7f0-067cc02f5327 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:36.542644 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12be6fc0-340f-4565-8740-b2b6aecf6c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:36.543507 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12be6fc0-340f-4565-8740-b2b6aecf6c3c None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:36.544028 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12be6fc0-340f-4565-8740-b2b6aecf6c3c None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:36.544575 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12be6fc0-340f-4565-8740-b2b6aecf6c3c None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:36.545479 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12be6fc0-340f-4565-8740-b2b6aecf6c3c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:36.545956 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12be6fc0-340f-4565-8740-b2b6aecf6c3c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:36.547577 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12be6fc0-340f-4565-8740-b2b6aecf6c3c None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:36.566548 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 288/563] 10.222.0.26 () {56 vars in 1133 bytes} [Wed Nov 2 03:21:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:36.572849 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12be6fc0-340f-4565-8740-b2b6aecf6c3c None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:36.592667 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991cf328-5b7e-438b-b0ea-927261b45d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:36.593300 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991cf328-5b7e-438b-b0ea-927261b45d88 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:36.593581 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991cf328-5b7e-438b-b0ea-927261b45d88 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:36.593855 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991cf328-5b7e-438b-b0ea-927261b45d88 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:36.594598 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991cf328-5b7e-438b-b0ea-927261b45d88 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:36.594908 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991cf328-5b7e-438b-b0ea-927261b45d88 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:36.596379 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991cf328-5b7e-438b-b0ea-927261b45d88 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:36.632079 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 276/564] 10.222.0.26 () {56 vars in 1149 bytes} [Wed Nov 2 03:21:36 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 435 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:36.637353 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991cf328-5b7e-438b-b0ea-927261b45d88 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:36.750385 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991cf328-5b7e-438b-b0ea-927261b45d88 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:36.769954 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12be6fc0-340f-4565-8740-b2b6aecf6c3c None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:36.791316 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d10eaa-8246-4f0f-b217-996c36c0a80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:36.792526 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d10eaa-8246-4f0f-b217-996c36c0a80e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:36.793044 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d10eaa-8246-4f0f-b217-996c36c0a80e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:36.793799 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d10eaa-8246-4f0f-b217-996c36c0a80e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:36.805336 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22481fff-839e-4249-8469-61be726fa8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:36.805940 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22481fff-839e-4249-8469-61be726fa8b2 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:36.806215 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22481fff-839e-4249-8469-61be726fa8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:36.806508 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22481fff-839e-4249-8469-61be726fa8b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:36.807775 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d10eaa-8246-4f0f-b217-996c36c0a80e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:36.809545 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d10eaa-8246-4f0f-b217-996c36c0a80e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:36.826875 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-c4d10eaa-8246-4f0f-b217-996c36c0a80e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:36.826875 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:36.826875 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:36.826875 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:36.826875 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:36.826875 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:36.826875 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:36.826875 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:36.826875 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:36.826875 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:36.826875 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:36.826875 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:36.826875 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 03:21:36.826875 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 03:21:36.826875 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 03:21:36.826875 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 03:21:36.827848 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:36.827848 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:36.827848 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:36.827848 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:36.827848 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:36.827848 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:36.827848 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:36.827848 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:36.827848 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:36.827848 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:36.827848 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:36.827848 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:36.827848 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:36.827848 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:36.827848 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:36.827848 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:36.827848 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 03:21:36.829141 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 03:21:36.829141 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:36.829141 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:36.831048 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 277/565] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:21:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 03:21:36.831834 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:36.836841 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d10eaa-8246-4f0f-b217-996c36c0a80e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:36.851558 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:36.881784 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22481fff-839e-4249-8469-61be726fa8b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:36.883251 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22481fff-839e-4249-8469-61be726fa8b2 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:36.961709 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d10eaa-8246-4f0f-b217-996c36c0a80e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:36.990764 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 289/566] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Nov 2 03:21:36 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Nov 02 03:21:37.002346 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22481fff-839e-4249-8469-61be726fa8b2 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:37.005133 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f731bdb-9d33-4772-be09-8534a1ba008c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:37.006056 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f731bdb-9d33-4772-be09-8534a1ba008c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:37.006554 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f731bdb-9d33-4772-be09-8534a1ba008c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:37.007044 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f731bdb-9d33-4772-be09-8534a1ba008c None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:37.007874 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f731bdb-9d33-4772-be09-8534a1ba008c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:37.008408 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f731bdb-9d33-4772-be09-8534a1ba008c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:37.010057 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f731bdb-9d33-4772-be09-8534a1ba008c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:37.023528 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 278/567] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Nov 2 03:21:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:37.028641 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f731bdb-9d33-4772-be09-8534a1ba008c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:37.107969 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:37.131910 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:37.166614 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f731bdb-9d33-4772-be09-8534a1ba008c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:37.192793 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22481fff-839e-4249-8469-61be726fa8b2 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:37.234069 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983a4288-3925-48c6-bc5e-09f1d8ad40d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:37.234717 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983a4288-3925-48c6-bc5e-09f1d8ad40d9 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:37.234993 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983a4288-3925-48c6-bc5e-09f1d8ad40d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:37.235261 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983a4288-3925-48c6-bc5e-09f1d8ad40d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:37.255116 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983a4288-3925-48c6-bc5e-09f1d8ad40d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:37.258230 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983a4288-3925-48c6-bc5e-09f1d8ad40d9 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:37.261079 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bfd115-e90e-4dff-b40a-ee19562e8141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:37.261984 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bfd115-e90e-4dff-b40a-ee19562e8141 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:37.262551 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bfd115-e90e-4dff-b40a-ee19562e8141 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:37.263023 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bfd115-e90e-4dff-b40a-ee19562e8141 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:37.276842 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bfd115-e90e-4dff-b40a-ee19562e8141 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:37.278304 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bfd115-e90e-4dff-b40a-ee19562e8141 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:37.279034 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-983a4288-3925-48c6-bc5e-09f1d8ad40d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:37.279034 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:37.279034 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:37.279034 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:37.279034 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:37.279034 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:37.279034 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:37.279034 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:37.279034 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:37.279034 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:37.279034 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:37.279034 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:37.279034 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 03:21:37.279034 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 03:21:37.279034 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 03:21:37.279034 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 03:21:37.280046 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:37.280046 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:37.280046 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:37.280046 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:37.280046 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:37.280046 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:37.280046 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:37.280046 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:37.280046 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:37.280046 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:37.280046 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:37.280046 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:37.280046 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:37.280046 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:37.280046 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:37.280046 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:37.280046 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 03:21:37.280993 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 03:21:37.280993 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:37.280993 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:37.280993 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 290/568] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:21:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 03:21:37.289327 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983a4288-3925-48c6-bc5e-09f1d8ad40d9 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:37.313718 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-a3bfd115-e90e-4dff-b40a-ee19562e8141 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 03:21:37.313718 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:37.313718 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:37.313718 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:37.313718 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:37.313718 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:37.313718 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:37.313718 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:37.313718 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:37.313718 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:37.313718 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:37.313718 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:37.313718 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 03:21:37.313718 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 03:21:37.313718 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 03:21:37.314786 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 03:21:37.314786 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:37.314786 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:37.314786 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:37.314786 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:37.314786 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:37.314786 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:37.314786 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:37.314786 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:37.314786 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:37.314786 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:37.314786 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:37.314786 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:37.314786 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:37.314786 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:37.314786 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:37.314786 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:37.315687 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 03:21:37.315687 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 03:21:37.315687 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 03:21:37.315687 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 03:21:37.315687 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 03:21:37.315687 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 03:21:37.315687 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 03:21:37.315687 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:37.315687 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 279/569] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Nov 2 03:21:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:37.320891 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bfd115-e90e-4dff-b40a-ee19562e8141 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:37.406469 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983a4288-3925-48c6-bc5e-09f1d8ad40d9 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:37.415121 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:37.445051 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6528bd1e-1bf4-4e63-9612-ddb1b9ac6aed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:37.446048 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6528bd1e-1bf4-4e63-9612-ddb1b9ac6aed None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:37.446605 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6528bd1e-1bf4-4e63-9612-ddb1b9ac6aed None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:37.447108 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6528bd1e-1bf4-4e63-9612-ddb1b9ac6aed None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:37.448046 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6528bd1e-1bf4-4e63-9612-ddb1b9ac6aed None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:37.448570 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6528bd1e-1bf4-4e63-9612-ddb1b9ac6aed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:37.450153 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6528bd1e-1bf4-4e63-9612-ddb1b9ac6aed None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:37.463150 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 291/570] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Nov 2 03:21:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:37.469107 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6528bd1e-1bf4-4e63-9612-ddb1b9ac6aed None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:37.520288 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bfd115-e90e-4dff-b40a-ee19562e8141 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:37.558739 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:37.567694 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe838387-e173-4284-94da-26dd52f644c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:37.568762 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe838387-e173-4284-94da-26dd52f644c2 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:37.570075 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe838387-e173-4284-94da-26dd52f644c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:37.570543 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe838387-e173-4284-94da-26dd52f644c2 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:37.571747 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe838387-e173-4284-94da-26dd52f644c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:37.572434 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe838387-e173-4284-94da-26dd52f644c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:37.574705 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe838387-e173-4284-94da-26dd52f644c2 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:37.579005 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6528bd1e-1bf4-4e63-9612-ddb1b9ac6aed None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:37.590798 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 280/571] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Nov 2 03:21:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:21:37.592649 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd067a-d280-45b9-b978-e942a0c24b6e None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:37.593314 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd067a-d280-45b9-b978-e942a0c24b6e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:37.593792 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedd067a-d280-45b9-b978-e942a0c24b6e None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:37.596342 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 281/572] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:21:37.597219 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedd067a-d280-45b9-b978-e942a0c24b6e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:37.621663 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0284c9-5d12-4177-9baa-e2f4eea6d5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:37.622508 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0284c9-5d12-4177-9baa-e2f4eea6d5f8 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:37.622889 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0284c9-5d12-4177-9baa-e2f4eea6d5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:37.623234 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0284c9-5d12-4177-9baa-e2f4eea6d5f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:37.631686 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0284c9-5d12-4177-9baa-e2f4eea6d5f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:37.633531 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0284c9-5d12-4177-9baa-e2f4eea6d5f8 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:37.646892 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-fa0284c9-5d12-4177-9baa-e2f4eea6d5f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 03:21:37.646892 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:37.646892 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:37.646892 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:37.646892 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:37.646892 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:37.646892 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:37.646892 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:37.646892 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:37.646892 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:37.646892 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:37.646892 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:37.646892 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 03:21:37.646892 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 03:21:37.646892 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 03:21:37.648476 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 03:21:37.648476 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:37.648476 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:37.648476 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:37.648476 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:37.648476 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:37.648476 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:37.648476 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:37.648476 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:37.648476 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:37.648476 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:37.648476 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:37.648476 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:37.648476 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:37.648476 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:37.648476 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:37.648476 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:37.649374 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 03:21:37.649374 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 03:21:37.649374 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 03:21:37.649374 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 03:21:37.649374 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 03:21:37.649374 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 03:21:37.649374 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 03:21:37.649374 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:37.652882 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 292/573] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Nov 2 03:21:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 03:21:37.654236 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41820b70-62b1-4fa3-9979-a7d43e8c3a9a None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:37.654518 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41820b70-62b1-4fa3-9979-a7d43e8c3a9a None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:37.654793 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41820b70-62b1-4fa3-9979-a7d43e8c3a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:37.677703 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-41820b70-62b1-4fa3-9979-a7d43e8c3a9a None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:37.679978 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-41820b70-62b1-4fa3-9979-a7d43e8c3a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:37.708047 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:37.731314 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedd067a-d280-45b9-b978-e942a0c24b6e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:37.765995 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 293/574] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 03:21:37.767150 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41820b70-62b1-4fa3-9979-a7d43e8c3a9a None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:37.768836 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1228108c-127e-43eb-a885-1fca515a06df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:37.769842 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228108c-127e-43eb-a885-1fca515a06df None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:37.770384 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228108c-127e-43eb-a885-1fca515a06df None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:37.770860 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1228108c-127e-43eb-a885-1fca515a06df None admin] PATH_INFO: `/v3/projects/d0b5e7017ac64ae6ab62c1a05aa54b72/users/da7ee607b66e43f3823d9d2faf373974/roles/e2a148d6fd534007bbb44d586d4306d1` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:37.787263 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1228108c-127e-43eb-a885-1fca515a06df None admin] RBAC: Authorizing `identity:create_grant(project_id=d0b5e7017ac64ae6ab62c1a05aa54b72, user_id=da7ee607b66e43f3823d9d2faf373974, role_id=e2a148d6fd534007bbb44d586d4306d1)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:37.789780 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1228108c-127e-43eb-a885-1fca515a06df None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:37.808415 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 282/575] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Nov 2 03:21:37 2022] PUT /identity/v3/projects/d0b5e7017ac64ae6ab62c1a05aa54b72/users/da7ee607b66e43f3823d9d2faf373974/roles/e2a148d6fd534007bbb44d586d4306d1 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:21:37.809982 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77d6cb1-4587-494a-96f3-17e6292d85de None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:37.810490 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77d6cb1-4587-494a-96f3-17e6292d85de None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:37.811034 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77d6cb1-4587-494a-96f3-17e6292d85de None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:37.813642 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 283/576] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:21:37.823051 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a14fc2-a45e-4f0c-a0f0-23e96c02635e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:37.823051 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a14fc2-a45e-4f0c-a0f0-23e96c02635e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:37.823051 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a14fc2-a45e-4f0c-a0f0-23e96c02635e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:37.858250 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-35a14fc2-a45e-4f0c-a0f0-23e96c02635e None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:37.859543 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-35a14fc2-a45e-4f0c-a0f0-23e96c02635e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:37.929594 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41820b70-62b1-4fa3-9979-a7d43e8c3a9a None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:37.944396 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:37.995330 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 284/577] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 03:21:37.998388 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad5cdcf-d6b8-450e-ba18-f1d60afe1e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:37.999006 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad5cdcf-d6b8-450e-ba18-f1d60afe1e55 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:37.999287 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad5cdcf-d6b8-450e-ba18-f1d60afe1e55 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:37.999565 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad5cdcf-d6b8-450e-ba18-f1d60afe1e55 None admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:38.000295 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad5cdcf-d6b8-450e-ba18-f1d60afe1e55 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:38.000601 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad5cdcf-d6b8-450e-ba18-f1d60afe1e55 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:38.002044 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad5cdcf-d6b8-450e-ba18-f1d60afe1e55 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:38.014963 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 294/578] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Nov 2 03:21:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:38.020809 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a14fc2-a45e-4f0c-a0f0-23e96c02635e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:38.054837 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad5cdcf-d6b8-450e-ba18-f1d60afe1e55 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:38.077514 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:38.137486 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a14fc2-a45e-4f0c-a0f0-23e96c02635e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:38.192754 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad5cdcf-d6b8-450e-ba18-f1d60afe1e55 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:38.195988 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c232a81-c718-4530-bc85-b9ba89ba6fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:38.196631 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c232a81-c718-4530-bc85-b9ba89ba6fe0 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:38.196916 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c232a81-c718-4530-bc85-b9ba89ba6fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:38.197193 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c232a81-c718-4530-bc85-b9ba89ba6fe0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:38.197928 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c232a81-c718-4530-bc85-b9ba89ba6fe0 None admin] RBAC: Adding query filter params (role.id=addc0db6b4ac4362b07fe21e14f89180, scope.project.id=d0b5e7017ac64ae6ab62c1a05aa54b72, user.id=44961b06151c4da382fcd54dd1f1e146) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:38.198237 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c232a81-c718-4530-bc85-b9ba89ba6fe0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:38.199763 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c232a81-c718-4530-bc85-b9ba89ba6fe0 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:38.211230 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 285/579] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Nov 2 03:21:38 2022] GET /identity/v3/role_assignments?user.id=44961b06151c4da382fcd54dd1f1e146&scope.project.id=d0b5e7017ac64ae6ab62c1a05aa54b72&role.id=addc0db6b4ac4362b07fe21e14f89180 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:21:38.261772 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1e518b-fa69-4e49-88d5-b4f646f63d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:38.262498 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e518b-fa69-4e49-88d5-b4f646f63d9f None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:38.262775 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e518b-fa69-4e49-88d5-b4f646f63d9f None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:38.263041 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e518b-fa69-4e49-88d5-b4f646f63d9f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:38.263629 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1e518b-fa69-4e49-88d5-b4f646f63d9f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:38.265068 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1e518b-fa69-4e49-88d5-b4f646f63d9f None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:38.281793 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-ea1e518b-fa69-4e49-88d5-b4f646f63d9f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 03:21:38.281793 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:38.281793 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 03:21:38.281793 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 03:21:38.281793 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 03:21:38.281793 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 03:21:38.281793 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 03:21:38.281793 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 03:21:38.281793 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 03:21:38.281793 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:38.281793 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 03:21:38.281793 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:38.281793 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:38.281793 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:38.281793 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:38.281793 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:38.281793 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:38.284196 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:38.284196 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:38.284196 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:38.284196 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:38.284196 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:38.284196 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:38.284196 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 03:21:38.284196 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 03:21:38.284196 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 03:21:38.284196 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 03:21:38.284196 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:38.284196 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:38.284196 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:38.284196 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:38.284196 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:38.284196 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:38.286312 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:38.286312 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:38.286312 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:38.286312 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:38.286312 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:38.286312 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:38.286312 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:38.286312 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:38.286312 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:38.286312 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:38.286312 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 03:21:38.286312 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 03:21:38.286312 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 03:21:38.286312 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:38.286312 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 295/580] 10.222.0.26 () {56 vars in 1129 bytes} [Wed Nov 2 03:21:38 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 03:21:38.295993 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c232a81-c718-4530-bc85-b9ba89ba6fe0 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:38.328223 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:38.377391 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1e518b-fa69-4e49-88d5-b4f646f63d9f None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:38.463568 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c232a81-c718-4530-bc85-b9ba89ba6fe0 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:38.501881 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f9fe11-6289-4a80-9d29-e35fbbb75ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:38.502497 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f9fe11-6289-4a80-9d29-e35fbbb75ed6 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:38.502774 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f9fe11-6289-4a80-9d29-e35fbbb75ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:38.503045 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f9fe11-6289-4a80-9d29-e35fbbb75ed6 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:38.503675 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f9fe11-6289-4a80-9d29-e35fbbb75ed6 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:38.504769 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f9fe11-6289-4a80-9d29-e35fbbb75ed6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:38.506141 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f9fe11-6289-4a80-9d29-e35fbbb75ed6 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:38.512211 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 286/581] 10.222.0.26 () {56 vars in 1124 bytes} [Wed Nov 2 03:21:38 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:38.515891 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:38.517728 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f9fe11-6289-4a80-9d29-e35fbbb75ed6 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:38.546741 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1e518b-fa69-4e49-88d5-b4f646f63d9f None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:38.589047 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d74ced-4290-4dc2-881e-d3e07fb5fd83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:38.590014 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d74ced-4290-4dc2-881e-d3e07fb5fd83 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:38.590914 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d74ced-4290-4dc2-881e-d3e07fb5fd83 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:38.591390 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d74ced-4290-4dc2-881e-d3e07fb5fd83 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:38.605114 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d74ced-4290-4dc2-881e-d3e07fb5fd83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:21:38.610709 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-e8d74ced-4290-4dc2-881e-d3e07fb5fd83 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 03:21:38.610709 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:38.610709 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:38.610709 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:38.610709 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:38.610709 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:38.610709 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:38.610709 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:38.610709 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:38.610709 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:38.610709 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:38.610709 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:38.610709 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:21:38.610709 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:21:38.610709 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:21:38.610709 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:21:38.611724 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:21:38.611724 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:21:38.611724 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:21:38.611724 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:21:38.611724 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:38.611724 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:38.611724 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:38.611724 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:38.611724 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:38.611724 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:38.611724 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:38.611724 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:38.611724 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:38.611724 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:38.611724 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:38.611724 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:38.612924 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:38.612924 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:38.612924 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:38.612924 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:38.612924 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:21:38.612924 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:21:38.612924 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:21:38.612924 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:21:38.612924 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:21:38.612924 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:21:38.612924 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 03:21:38.612924 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:38.615687 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 296/582] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Nov 2 03:21:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:38.617685 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a8d4c-5df3-4e2d-91c3-1d8c87b612fa None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:38.618155 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a8d4c-5df3-4e2d-91c3-1d8c87b612fa None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:38.618626 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513a8d4c-5df3-4e2d-91c3-1d8c87b612fa None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:38.620071 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 297/583] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:21:38.623047 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513a8d4c-5df3-4e2d-91c3-1d8c87b612fa None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:38.641396 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:38.645695 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f9fe11-6289-4a80-9d29-e35fbbb75ed6 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:38.685903 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d91404e-b251-4751-9018-44328f6d4ef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:38.686688 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d91404e-b251-4751-9018-44328f6d4ef9 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:38.687023 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d91404e-b251-4751-9018-44328f6d4ef9 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:38.687363 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d91404e-b251-4751-9018-44328f6d4ef9 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:38.688380 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d91404e-b251-4751-9018-44328f6d4ef9 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:38.688747 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d91404e-b251-4751-9018-44328f6d4ef9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:38.690946 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d91404e-b251-4751-9018-44328f6d4ef9 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:38.698540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 287/584] 10.222.0.26 () {56 vars in 1124 bytes} [Wed Nov 2 03:21:38 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:38.705779 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc0f6d2-39db-422b-9677-19b886ab6d42 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:38.706137 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc0f6d2-39db-422b-9677-19b886ab6d42 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:38.706500 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc0f6d2-39db-422b-9677-19b886ab6d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:38.727910 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-ffc0f6d2-39db-422b-9677-19b886ab6d42 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:38.729242 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ffc0f6d2-39db-422b-9677-19b886ab6d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:38.741672 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513a8d4c-5df3-4e2d-91c3-1d8c87b612fa None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:38.779747 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11a7f16-d9c7-43d5-9393-f94e4a873bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:38.780506 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a7f16-d9c7-43d5-9393-f94e4a873bfe None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:38.780845 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a7f16-d9c7-43d5-9393-f94e4a873bfe None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:38.781183 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11a7f16-d9c7-43d5-9393-f94e4a873bfe None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:38.781953 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11a7f16-d9c7-43d5-9393-f94e4a873bfe None admin] RBAC: Adding query filter params (name=admin) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:38.782298 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11a7f16-d9c7-43d5-9393-f94e4a873bfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:38.783724 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a11a7f16-d9c7-43d5-9393-f94e4a873bfe None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:38.793577 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 298/585] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Nov 2 03:21:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:21:38.794488 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11a7f16-d9c7-43d5-9393-f94e4a873bfe None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:38.797314 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:38.811328 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 288/586] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 03:21:38.812681 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc0f6d2-39db-422b-9677-19b886ab6d42 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:38.916315 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:38.925459 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc0f6d2-39db-422b-9677-19b886ab6d42 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:38.961431 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc677c-4b77-4f1c-9de7-221550e3df15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:38.962135 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc677c-4b77-4f1c-9de7-221550e3df15 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:38.962447 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc677c-4b77-4f1c-9de7-221550e3df15 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:38.962680 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7efc677c-4b77-4f1c-9de7-221550e3df15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:38.984752 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efc677c-4b77-4f1c-9de7-221550e3df15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:38.985021 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11a7f16-d9c7-43d5-9393-f94e4a873bfe None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:38.986223 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7efc677c-4b77-4f1c-9de7-221550e3df15 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:39.015015 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-7efc677c-4b77-4f1c-9de7-221550e3df15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:39.015015 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:39.015015 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:39.015015 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:39.015015 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:39.015015 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:39.015015 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:39.015015 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:39.015015 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:39.015015 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:39.015015 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:39.015015 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:39.015015 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 03:21:39.015015 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 03:21:39.015015 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 03:21:39.015015 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 03:21:39.016054 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:39.016054 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:39.016054 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:39.016054 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:39.016054 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:39.016054 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:39.016054 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:39.016054 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:39.016054 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:39.016054 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:39.016054 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:39.016054 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:39.016054 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:39.016054 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:39.016054 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:39.016054 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:39.016054 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 03:21:39.016952 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 03:21:39.016952 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:39.016952 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:39.025280 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 289/587] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:21:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:39.026101 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc677c-4b77-4f1c-9de7-221550e3df15 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:39.050019 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccb591f-54a0-4670-b60e-60fe056fdcc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:39.050563 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccb591f-54a0-4670-b60e-60fe056fdcc9 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:39.050842 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccb591f-54a0-4670-b60e-60fe056fdcc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:39.051109 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccb591f-54a0-4670-b60e-60fe056fdcc9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:39.051810 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccb591f-54a0-4670-b60e-60fe056fdcc9 None admin] RBAC: Adding query filter params (interface=public, service_id=ce2e2001480649a786733b3694f6d2e3, region_id=RegionOne) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:39.052149 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccb591f-54a0-4670-b60e-60fe056fdcc9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:39.054056 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccb591f-54a0-4670-b60e-60fe056fdcc9 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:39.070273 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 299/588] 10.222.0.26 () {56 vars in 1251 bytes} [Wed Nov 2 03:21:38 2022] GET /identity/v3/endpoints?service_id=ce2e2001480649a786733b3694f6d2e3&interface=public®ion_id=RegionOne => generated 194 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:39.075398 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccb591f-54a0-4670-b60e-60fe056fdcc9 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:39.190620 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:39.208977 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7efc677c-4b77-4f1c-9de7-221550e3df15 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:39.246903 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366feff0-88a1-48f9-bbbe-ddea22597a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:39.247523 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366feff0-88a1-48f9-bbbe-ddea22597a6f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:39.247799 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366feff0-88a1-48f9-bbbe-ddea22597a6f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:39.248064 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccb591f-54a0-4670-b60e-60fe056fdcc9 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:39.248186 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366feff0-88a1-48f9-bbbe-ddea22597a6f None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:39.248731 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366feff0-88a1-48f9-bbbe-ddea22597a6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:39.249032 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366feff0-88a1-48f9-bbbe-ddea22597a6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:39.250396 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366feff0-88a1-48f9-bbbe-ddea22597a6f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:39.267406 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 290/589] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Nov 2 03:21:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:39.268647 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366feff0-88a1-48f9-bbbe-ddea22597a6f None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:39.273082 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:39.292988 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8a6901-6fcc-4587-bfa5-def77a33b428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:39.294565 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a6901-6fcc-4587-bfa5-def77a33b428 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:39.295051 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a6901-6fcc-4587-bfa5-def77a33b428 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:39.295556 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8a6901-6fcc-4587-bfa5-def77a33b428 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:39.296374 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8a6901-6fcc-4587-bfa5-def77a33b428 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:39.298596 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8a6901-6fcc-4587-bfa5-def77a33b428 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:39.306194 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 300/590] 10.222.0.26 () {56 vars in 1087 bytes} [Wed Nov 2 03:21:39 2022] GET /identity/v3/services => generated 772 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:39.306939 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8a6901-6fcc-4587-bfa5-def77a33b428 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:39.408334 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366feff0-88a1-48f9-bbbe-ddea22597a6f None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:39.430115 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8a6901-6fcc-4587-bfa5-def77a33b428 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:39.459512 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7802521-5405-43a4-93a0-71492a7134dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:39.461634 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7802521-5405-43a4-93a0-71492a7134dc None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:39.462180 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7802521-5405-43a4-93a0-71492a7134dc None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:39.462611 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7802521-5405-43a4-93a0-71492a7134dc None admin] PATH_INFO: `/v3/services/network` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:39.463627 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:39.464422 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7802521-5405-43a4-93a0-71492a7134dc None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:39.467614 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7802521-5405-43a4-93a0-71492a7134dc None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:39.471305 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0119328c-3f1b-4eb3-b2cc-1f3aae008016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:39.472156 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0119328c-3f1b-4eb3-b2cc-1f3aae008016 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:39.472535 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0119328c-3f1b-4eb3-b2cc-1f3aae008016 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:39.472907 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0119328c-3f1b-4eb3-b2cc-1f3aae008016 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:39.480130 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-e7802521-5405-43a4-93a0-71492a7134dc None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 03:21:39.480130 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:39.480130 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 03:21:39.480130 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 03:21:39.480130 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 03:21:39.480130 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 03:21:39.480130 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 03:21:39.480130 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 03:21:39.480130 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 03:21:39.480130 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:39.480130 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 03:21:39.480130 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:39.480130 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:39.480130 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:39.480130 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:39.480130 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:39.480130 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:39.481379 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:39.481379 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:39.481379 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:39.481379 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:39.481379 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:39.481379 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:39.481379 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 03:21:39.481379 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 03:21:39.481379 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 03:21:39.481379 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 03:21:39.481379 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:39.481379 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:39.481379 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:39.481379 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:39.481379 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:39.481379 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:39.482440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:39.482440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:39.482440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:39.482440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:39.482440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:39.482440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:39.482440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:39.482440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:39.482440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:39.482440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:39.482440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 03:21:39.482440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 03:21:39.482440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 03:21:39.482440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:39.484683 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0119328c-3f1b-4eb3-b2cc-1f3aae008016 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:39.490369 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0119328c-3f1b-4eb3-b2cc-1f3aae008016 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:39.490560 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 291/591] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Nov 2 03:21:39 2022] GET /identity/v3/services/network => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:39.492978 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7802521-5405-43a4-93a0-71492a7134dc None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:39.505116 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-0119328c-3f1b-4eb3-b2cc-1f3aae008016 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Nov 02 03:21:39.505116 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:39.505116 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:39.505116 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:39.505116 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:39.505116 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:39.505116 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:39.505116 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:39.505116 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:39.505116 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:39.505116 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:39.505116 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:39.505116 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 03:21:39.505116 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 03:21:39.505116 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 03:21:39.505116 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 03:21:39.506199 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:39.506199 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:39.506199 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 03:21:39.506199 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 03:21:39.506199 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 03:21:39.506199 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 03:21:39.506199 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:39.506199 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:39.506199 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:39.506199 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:39.506199 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:39.506199 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:39.506199 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:39.506199 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:39.506199 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:39.506199 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:39.506199 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:39.506199 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:39.507248 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:39.507248 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:39.507248 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 03:21:39.507248 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 03:21:39.507248 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 03:21:39.507248 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 03:21:39.507248 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 03:21:39.507248 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 03:21:39.507248 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Nov 02 03:21:39.507248 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:39.507248 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 301/592] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Nov 2 03:21:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:39.519987 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0119328c-3f1b-4eb3-b2cc-1f3aae008016 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:39.637787 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:39.677825 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0119328c-3f1b-4eb3-b2cc-1f3aae008016 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:39.683204 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7802521-5405-43a4-93a0-71492a7134dc None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:39.716822 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d381f9bc-e9c4-4dea-bfcd-6964f40a718f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:39.717078 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d381f9bc-e9c4-4dea-bfcd-6964f40a718f None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:39.717405 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d381f9bc-e9c4-4dea-bfcd-6964f40a718f None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:39.717615 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d381f9bc-e9c4-4dea-bfcd-6964f40a718f None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:39.718319 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d381f9bc-e9c4-4dea-bfcd-6964f40a718f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:39.718621 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d381f9bc-e9c4-4dea-bfcd-6964f40a718f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:39.721489 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d381f9bc-e9c4-4dea-bfcd-6964f40a718f None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:39.741268 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 302/593] 10.222.0.26 () {56 vars in 1135 bytes} [Wed Nov 2 03:21:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:39.746794 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d381f9bc-e9c4-4dea-bfcd-6964f40a718f None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:39.765226 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99e6fa5-443e-4d06-bc0f-21907aa3f3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:39.765855 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e6fa5-443e-4d06-bc0f-21907aa3f3a4 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:39.766136 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e6fa5-443e-4d06-bc0f-21907aa3f3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:39.766412 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99e6fa5-443e-4d06-bc0f-21907aa3f3a4 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:39.767044 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99e6fa5-443e-4d06-bc0f-21907aa3f3a4 None admin] RBAC: Adding query filter params (name=network) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:39.767353 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99e6fa5-443e-4d06-bc0f-21907aa3f3a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:39.768755 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99e6fa5-443e-4d06-bc0f-21907aa3f3a4 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:39.786183 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 292/594] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Nov 2 03:21:39 2022] GET /identity/v3/services?name=network => generated 124 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:21:39.799975 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99e6fa5-443e-4d06-bc0f-21907aa3f3a4 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:39.810799 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:39.922178 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d381f9bc-e9c4-4dea-bfcd-6964f40a718f None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:39.935283 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99e6fa5-443e-4d06-bc0f-21907aa3f3a4 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:39.971156 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86b0180-9ffc-43dc-9b04-92fb1fb088f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:39.971773 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b0180-9ffc-43dc-9b04-92fb1fb088f1 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:39.972087 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b0180-9ffc-43dc-9b04-92fb1fb088f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:39.972368 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86b0180-9ffc-43dc-9b04-92fb1fb088f1 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:39.973154 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86b0180-9ffc-43dc-9b04-92fb1fb088f1 None admin] RBAC: Adding query filter params (type=network) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:39.973591 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86b0180-9ffc-43dc-9b04-92fb1fb088f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:39.974987 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86b0180-9ffc-43dc-9b04-92fb1fb088f1 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:39.977582 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:39.980990 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 293/595] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Nov 2 03:21:39 2022] GET /identity/v3/services?type=network => generated 350 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:39.987213 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86b0180-9ffc-43dc-9b04-92fb1fb088f1 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:40.024686 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b2379-c8fd-40b1-b5cb-4c9e8d2b1dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:40.025312 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b2379-c8fd-40b1-b5cb-4c9e8d2b1dac None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:40.025614 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b2379-c8fd-40b1-b5cb-4c9e8d2b1dac None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:40.025861 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b2379-c8fd-40b1-b5cb-4c9e8d2b1dac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:40.122370 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822b2379-c8fd-40b1-b5cb-4c9e8d2b1dac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:40.123826 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822b2379-c8fd-40b1-b5cb-4c9e8d2b1dac None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:40.188060 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86b0180-9ffc-43dc-9b04-92fb1fb088f1 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:40.213833 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:40.230919 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade6e0cb-b051-42f4-9270-95b95b297463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:40.232263 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade6e0cb-b051-42f4-9270-95b95b297463 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:40.232751 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade6e0cb-b051-42f4-9270-95b95b297463 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:40.233303 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade6e0cb-b051-42f4-9270-95b95b297463 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:40.234207 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade6e0cb-b051-42f4-9270-95b95b297463 None admin] RBAC: Adding query filter params (interface=public, service_id=023cee57971f4454962faf2670021920, region_id=RegionOne) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:40.234728 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade6e0cb-b051-42f4-9270-95b95b297463 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:40.236320 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade6e0cb-b051-42f4-9270-95b95b297463 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:40.247096 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 294/596] 10.222.0.26 () {56 vars in 1251 bytes} [Wed Nov 2 03:21:39 2022] GET /identity/v3/endpoints?service_id=023cee57971f4454962faf2670021920&interface=public®ion_id=RegionOne => generated 194 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:40.252482 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade6e0cb-b051-42f4-9270-95b95b297463 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:40.327256 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:40.335593 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 303/597] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Nov 2 03:21:39 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 589 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Nov 02 03:21:40.339732 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51319aa-070e-4445-9786-74e7eb3e6573 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:40.339732 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51319aa-070e-4445-9786-74e7eb3e6573 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:40.339732 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51319aa-070e-4445-9786-74e7eb3e6573 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:40.342603 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 304/598] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:21:40.342911 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51319aa-070e-4445-9786-74e7eb3e6573 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:40.384972 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade6e0cb-b051-42f4-9270-95b95b297463 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:40.421137 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821ba07e-8319-4c18-82fc-1115a1fdc4cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:40.421756 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ba07e-8319-4c18-82fc-1115a1fdc4cf None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:40.422032 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ba07e-8319-4c18-82fc-1115a1fdc4cf None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:40.422354 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ba07e-8319-4c18-82fc-1115a1fdc4cf None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:40.422960 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821ba07e-8319-4c18-82fc-1115a1fdc4cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:40.424450 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821ba07e-8319-4c18-82fc-1115a1fdc4cf None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:40.430091 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 295/599] 10.222.0.26 () {56 vars in 1087 bytes} [Wed Nov 2 03:21:40 2022] GET /identity/v3/services => generated 772 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:21:40.431281 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b581807a-2439-4a0f-a4a8-05dee1103fa9 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:40.431566 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b581807a-2439-4a0f-a4a8-05dee1103fa9 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:40.431859 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b581807a-2439-4a0f-a4a8-05dee1103fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:40.455264 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-b581807a-2439-4a0f-a4a8-05dee1103fa9 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:40.456505 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b581807a-2439-4a0f-a4a8-05dee1103fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:40.488045 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51319aa-070e-4445-9786-74e7eb3e6573 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:40.549412 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94299b3d-7a0f-47c9-b1aa-ff9824cf6bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:40.551769 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94299b3d-7a0f-47c9-b1aa-ff9824cf6bf5 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:40.552123 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94299b3d-7a0f-47c9-b1aa-ff9824cf6bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:40.552401 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94299b3d-7a0f-47c9-b1aa-ff9824cf6bf5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:40.577813 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94299b3d-7a0f-47c9-b1aa-ff9824cf6bf5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:40.579302 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94299b3d-7a0f-47c9-b1aa-ff9824cf6bf5 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:40.596281 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 296/600] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 03:21:40.599997 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:40.608533 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-94299b3d-7a0f-47c9-b1aa-ff9824cf6bf5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:40.608533 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:40.608533 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:40.608533 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:40.608533 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:40.608533 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:40.608533 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:40.608533 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:40.608533 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:40.608533 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:40.608533 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:40.608533 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:40.608533 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 03:21:40.608533 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 03:21:40.608533 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 03:21:40.608533 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 03:21:40.609575 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:40.609575 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:40.609575 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:40.609575 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:40.609575 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:40.609575 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:40.609575 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:40.609575 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:40.609575 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:40.609575 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:40.609575 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:40.609575 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:40.609575 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:40.609575 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:40.609575 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:40.609575 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:40.609575 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 03:21:40.610502 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 03:21:40.610502 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:40.610502 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:40.610502 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 305/601] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:21:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:40.617553 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b581807a-2439-4a0f-a4a8-05dee1103fa9 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:40.684538 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:40.730582 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b581807a-2439-4a0f-a4a8-05dee1103fa9 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:40.770458 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52e8f7e-efe4-405a-bdd6-98a4a7729d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:40.771090 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52e8f7e-efe4-405a-bdd6-98a4a7729d63 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:40.771369 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52e8f7e-efe4-405a-bdd6-98a4a7729d63 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:40.771642 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52e8f7e-efe4-405a-bdd6-98a4a7729d63 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:40.772876 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52e8f7e-efe4-405a-bdd6-98a4a7729d63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:40.773179 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52e8f7e-efe4-405a-bdd6-98a4a7729d63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:40.774548 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52e8f7e-efe4-405a-bdd6-98a4a7729d63 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:40.787111 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 297/602] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Nov 2 03:21:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:40.792716 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94299b3d-7a0f-47c9-b1aa-ff9824cf6bf5 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:40.868051 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6983225-5b54-42fd-a422-367ae34026d9 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:40.868368 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6983225-5b54-42fd-a422-367ae34026d9 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:40.868648 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6983225-5b54-42fd-a422-367ae34026d9 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:40.869978 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 298/603] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:21:40.875097 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924d8b1-58ca-43d2-a782-cc679b759e66 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:40.875378 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924d8b1-58ca-43d2-a782-cc679b759e66 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:40.875646 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1924d8b1-58ca-43d2-a782-cc679b759e66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:40.895214 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-1924d8b1-58ca-43d2-a782-cc679b759e66 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:40.896427 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1924d8b1-58ca-43d2-a782-cc679b759e66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:40.912958 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94299b3d-7a0f-47c9-b1aa-ff9824cf6bf5 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:40.954415 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebae42e7-1d47-4ec4-9783-12ac2795b6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:40.955054 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae42e7-1d47-4ec4-9783-12ac2795b6a6 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:40.955335 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae42e7-1d47-4ec4-9783-12ac2795b6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:40.955607 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebae42e7-1d47-4ec4-9783-12ac2795b6a6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:40.965064 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebae42e7-1d47-4ec4-9783-12ac2795b6a6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:40.967320 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebae42e7-1d47-4ec4-9783-12ac2795b6a6 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:40.983626 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-ebae42e7-1d47-4ec4-9783-12ac2795b6a6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 03:21:40.983626 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:40.983626 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:40.983626 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:40.983626 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:40.983626 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:40.983626 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:40.983626 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:40.983626 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:40.983626 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:40.983626 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:40.983626 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:40.983626 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 03:21:40.983626 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 03:21:40.983626 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 03:21:40.984578 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 03:21:40.984578 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:40.984578 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:40.984578 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:40.984578 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:40.984578 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:40.984578 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:40.984578 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:40.984578 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:40.984578 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:40.984578 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:40.984578 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:40.984578 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:40.984578 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:40.984578 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:40.984578 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:40.984578 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:40.985477 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 03:21:40.985477 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 03:21:40.985477 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 03:21:40.985477 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 03:21:40.985477 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 03:21:40.985477 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 03:21:40.985477 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 03:21:40.985477 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:40.985934 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 306/604] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Nov 2 03:21:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 03:21:40.987983 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebae42e7-1d47-4ec4-9783-12ac2795b6a6 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:40.988960 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 299/605] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 03:21:40.997297 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1924d8b1-58ca-43d2-a782-cc679b759e66 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:41.058234 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:41.092111 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:41.126892 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebae42e7-1d47-4ec4-9783-12ac2795b6a6 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:41.181334 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307e9960-6741-4bd9-80e4-7308b9d05105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:41.181972 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e9960-6741-4bd9-80e4-7308b9d05105 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:41.182251 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e9960-6741-4bd9-80e4-7308b9d05105 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:41.182520 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307e9960-6741-4bd9-80e4-7308b9d05105 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:41.194812 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307e9960-6741-4bd9-80e4-7308b9d05105 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:21:41.201723 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-307e9960-6741-4bd9-80e4-7308b9d05105 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 03:21:41.201723 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:41.201723 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:41.201723 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:41.201723 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:41.201723 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:41.201723 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:41.201723 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:41.201723 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:41.201723 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:41.201723 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:41.201723 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:41.201723 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:21:41.201723 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:21:41.201723 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:21:41.201723 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:21:41.202705 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:21:41.202705 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:21:41.202705 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:21:41.202705 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:21:41.202705 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:41.202705 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:41.202705 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:41.202705 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:41.202705 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:41.202705 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:41.202705 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:41.202705 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:41.202705 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:41.202705 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:41.202705 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:41.202705 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:41.203541 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:41.203541 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:41.203541 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:41.203541 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:41.203541 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:21:41.203541 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:21:41.203541 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:21:41.203541 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:21:41.203541 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:21:41.203541 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:21:41.203541 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 03:21:41.203541 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:41.203541 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 307/606] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Nov 2 03:21:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:41.210610 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307e9960-6741-4bd9-80e4-7308b9d05105 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:41.216880 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1924d8b1-58ca-43d2-a782-cc679b759e66 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:41.264235 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dee3d46-cad4-4716-9d23-eb80c0f26005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:41.266049 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee3d46-cad4-4716-9d23-eb80c0f26005 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:41.266734 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee3d46-cad4-4716-9d23-eb80c0f26005 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:41.267250 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dee3d46-cad4-4716-9d23-eb80c0f26005 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:41.268197 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dee3d46-cad4-4716-9d23-eb80c0f26005 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:41.268738 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dee3d46-cad4-4716-9d23-eb80c0f26005 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:41.270375 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dee3d46-cad4-4716-9d23-eb80c0f26005 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:41.294211 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 300/607] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Nov 2 03:21:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:21:41.300012 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dee3d46-cad4-4716-9d23-eb80c0f26005 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:41.331276 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307e9960-6741-4bd9-80e4-7308b9d05105 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:41.353888 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:41.371413 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873ef257-f4d5-4c0e-8b04-91459e8d743e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:41.372071 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ef257-f4d5-4c0e-8b04-91459e8d743e None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:41.372354 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ef257-f4d5-4c0e-8b04-91459e8d743e None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:41.372625 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873ef257-f4d5-4c0e-8b04-91459e8d743e None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:41.373290 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873ef257-f4d5-4c0e-8b04-91459e8d743e None admin] RBAC: Adding query filter params (name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:41.373596 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873ef257-f4d5-4c0e-8b04-91459e8d743e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:41.374962 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873ef257-f4d5-4c0e-8b04-91459e8d743e None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:41.383426 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 308/608] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Nov 2 03:21:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:21:41.388392 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873ef257-f4d5-4c0e-8b04-91459e8d743e None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:41.423152 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dee3d46-cad4-4716-9d23-eb80c0f26005 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:41.435938 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:41.467133 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c355793-d15d-43b8-974f-ba95dbd19fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:41.467932 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c355793-d15d-43b8-974f-ba95dbd19fb5 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:41.468334 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c355793-d15d-43b8-974f-ba95dbd19fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:41.468710 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c355793-d15d-43b8-974f-ba95dbd19fb5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:41.469544 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c355793-d15d-43b8-974f-ba95dbd19fb5 None admin] RBAC: Adding query filter params (role.id=addc0db6b4ac4362b07fe21e14f89180, scope.project.id=d0b5e7017ac64ae6ab62c1a05aa54b72, user.id=5193c40a917b4105ab8b27aead1685f8) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:41.469956 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c355793-d15d-43b8-974f-ba95dbd19fb5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:41.471550 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c355793-d15d-43b8-974f-ba95dbd19fb5 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:41.478850 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 301/609] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Nov 2 03:21:41 2022] GET /identity/v3/role_assignments?user.id=5193c40a917b4105ab8b27aead1685f8&scope.project.id=d0b5e7017ac64ae6ab62c1a05aa54b72&role.id=addc0db6b4ac4362b07fe21e14f89180 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:21:41.479847 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c355793-d15d-43b8-974f-ba95dbd19fb5 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:41.505373 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873ef257-f4d5-4c0e-8b04-91459e8d743e None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:41.590531 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e406d0f6-a554-488a-8f9b-d69fd6660660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:41.591208 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e406d0f6-a554-488a-8f9b-d69fd6660660 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:41.591453 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e406d0f6-a554-488a-8f9b-d69fd6660660 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:41.591723 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e406d0f6-a554-488a-8f9b-d69fd6660660 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:41.593298 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e406d0f6-a554-488a-8f9b-d69fd6660660 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:41.594710 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e406d0f6-a554-488a-8f9b-d69fd6660660 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:41.611429 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-e406d0f6-a554-488a-8f9b-d69fd6660660 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 03:21:41.611429 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:41.611429 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 03:21:41.611429 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 03:21:41.611429 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 03:21:41.611429 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 03:21:41.611429 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 03:21:41.611429 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 03:21:41.611429 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 03:21:41.611429 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:41.611429 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 03:21:41.611429 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:41.611429 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:41.611429 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:41.611429 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:41.611429 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:41.612758 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:41.612758 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:41.612758 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:41.612758 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:41.612758 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:41.612758 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:41.612758 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:41.612758 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 03:21:41.612758 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 03:21:41.612758 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 03:21:41.612758 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 03:21:41.612758 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:41.612758 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:41.612758 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:41.612758 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:41.612758 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:41.612758 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:41.613655 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:41.613655 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:41.613655 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:41.613655 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:41.613655 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:41.613655 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:41.613655 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:41.613655 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:41.613655 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:41.613655 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:41.613655 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 03:21:41.613655 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 03:21:41.613655 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 03:21:41.613655 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:41.613655 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 309/610] 10.222.0.26 () {56 vars in 1132 bytes} [Wed Nov 2 03:21:41 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:41.622857 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e406d0f6-a554-488a-8f9b-d69fd6660660 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:41.679710 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:41.710436 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c355793-d15d-43b8-974f-ba95dbd19fb5 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:41.741830 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e406d0f6-a554-488a-8f9b-d69fd6660660 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:41.750923 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a869ae-c112-4294-8946-185b1d93468c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:41.751928 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a869ae-c112-4294-8946-185b1d93468c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:41.752465 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a869ae-c112-4294-8946-185b1d93468c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:41.752939 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a869ae-c112-4294-8946-185b1d93468c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:41.766080 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a869ae-c112-4294-8946-185b1d93468c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:41.767846 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a869ae-c112-4294-8946-185b1d93468c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:41.778367 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d592bb03-efbc-4735-80bc-6d94b6978fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:41.778910 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592bb03-efbc-4735-80bc-6d94b6978fac None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:41.779195 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592bb03-efbc-4735-80bc-6d94b6978fac None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:41.779470 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d592bb03-efbc-4735-80bc-6d94b6978fac None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:41.780131 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d592bb03-efbc-4735-80bc-6d94b6978fac None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:41.780433 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d592bb03-efbc-4735-80bc-6d94b6978fac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:41.781788 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d592bb03-efbc-4735-80bc-6d94b6978fac None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:41.784822 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-11a869ae-c112-4294-8946-185b1d93468c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:41.784822 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:41.784822 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:41.784822 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:41.784822 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:41.784822 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:41.784822 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:41.784822 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:41.784822 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:41.784822 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:41.784822 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:41.784822 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:41.784822 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 03:21:41.784822 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 03:21:41.784822 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 03:21:41.784822 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 03:21:41.785749 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:41.785749 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:41.785749 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:41.785749 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:41.785749 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:41.785749 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:41.785749 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:41.785749 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:41.785749 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:41.785749 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:41.785749 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:41.785749 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:41.785749 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:41.785749 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:41.785749 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:41.785749 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:41.785749 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 03:21:41.786632 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 03:21:41.786632 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:41.786632 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:41.786812 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 310/611] 10.222.0.26 () {56 vars in 1126 bytes} [Wed Nov 2 03:21:41 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:41.789099 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 302/612] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:21:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:41.793569 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d592bb03-efbc-4735-80bc-6d94b6978fac None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:41.800290 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a869ae-c112-4294-8946-185b1d93468c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:41.829239 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:41.930945 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a869ae-c112-4294-8946-185b1d93468c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:41.969272 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81bddce-69e8-43d9-af67-dba2e6163c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:41.970255 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81bddce-69e8-43d9-af67-dba2e6163c39 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:41.970743 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81bddce-69e8-43d9-af67-dba2e6163c39 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:41.971219 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81bddce-69e8-43d9-af67-dba2e6163c39 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:41.972119 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81bddce-69e8-43d9-af67-dba2e6163c39 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:41.972789 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81bddce-69e8-43d9-af67-dba2e6163c39 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:41.974369 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81bddce-69e8-43d9-af67-dba2e6163c39 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:41.988822 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 303/613] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Nov 2 03:21:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:41.990091 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d592bb03-efbc-4735-80bc-6d94b6978fac None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:41.990645 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f75b3-6a83-488f-9832-3adfaa527dd0 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:41.991217 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f75b3-6a83-488f-9832-3adfaa527dd0 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:41.991704 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f75b3-6a83-488f-9832-3adfaa527dd0 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:41.993478 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 304/614] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:21:41.994430 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9f75b3-6a83-488f-9832-3adfaa527dd0 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:42.008327 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:42.029081 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d16d8a-0eee-47e3-96b7-50b02527cb53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:42.029837 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d16d8a-0eee-47e3-96b7-50b02527cb53 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:42.029951 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d16d8a-0eee-47e3-96b7-50b02527cb53 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:42.030167 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d16d8a-0eee-47e3-96b7-50b02527cb53 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:42.030798 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d16d8a-0eee-47e3-96b7-50b02527cb53 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:42.031146 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d16d8a-0eee-47e3-96b7-50b02527cb53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:42.032719 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d16d8a-0eee-47e3-96b7-50b02527cb53 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:42.038777 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 311/615] 10.222.0.26 () {56 vars in 1126 bytes} [Wed Nov 2 03:21:41 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:42.040089 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867af552-f48d-4616-85ef-f0cdd94902cb None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:42.040374 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867af552-f48d-4616-85ef-f0cdd94902cb None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:42.040652 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867af552-f48d-4616-85ef-f0cdd94902cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:42.059981 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-867af552-f48d-4616-85ef-f0cdd94902cb None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:42.061312 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-867af552-f48d-4616-85ef-f0cdd94902cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:42.093664 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:42.118452 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9f75b3-6a83-488f-9832-3adfaa527dd0 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:42.142525 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 312/616] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 03:21:42.161249 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1414faff-339e-4b28-aa6a-e9c8c7316605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:42.162275 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1414faff-339e-4b28-aa6a-e9c8c7316605 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:42.162839 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1414faff-339e-4b28-aa6a-e9c8c7316605 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:42.163334 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1414faff-339e-4b28-aa6a-e9c8c7316605 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:42.171652 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1414faff-339e-4b28-aa6a-e9c8c7316605 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:42.173444 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1414faff-339e-4b28-aa6a-e9c8c7316605 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:42.187425 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-1414faff-339e-4b28-aa6a-e9c8c7316605 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 03:21:42.187425 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:42.187425 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:42.187425 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:42.187425 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:42.187425 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:42.187425 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:42.187425 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:42.187425 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:42.187425 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:42.187425 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:42.187425 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:42.187425 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 03:21:42.187425 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 03:21:42.187425 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 03:21:42.187425 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 03:21:42.188461 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:42.188461 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:42.188461 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 03:21:42.188461 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 03:21:42.188461 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 03:21:42.188461 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 03:21:42.188461 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:42.188461 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:42.188461 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:42.188461 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:42.188461 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:42.188461 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:42.188461 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:42.188461 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:42.188461 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:42.188461 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:42.188461 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:42.188461 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:42.189422 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:42.189422 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:42.189422 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 03:21:42.189422 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 03:21:42.189422 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 03:21:42.189422 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 03:21:42.189422 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 03:21:42.189422 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 03:21:42.189422 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 03:21:42.189422 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:42.191971 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 305/617] 10.222.0.26 () {56 vars in 1117 bytes} [Wed Nov 2 03:21:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:42.197253 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867af552-f48d-4616-85ef-f0cdd94902cb None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:42.311451 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867af552-f48d-4616-85ef-f0cdd94902cb None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:42.313767 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:42.354798 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb056e1-6ea2-46b4-97c2-b89746af7b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:42.355687 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb056e1-6ea2-46b4-97c2-b89746af7b8c None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:42.356171 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb056e1-6ea2-46b4-97c2-b89746af7b8c None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:42.356593 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb056e1-6ea2-46b4-97c2-b89746af7b8c None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:42.357639 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb056e1-6ea2-46b4-97c2-b89746af7b8c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:42.358106 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb056e1-6ea2-46b4-97c2-b89746af7b8c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:42.359683 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb056e1-6ea2-46b4-97c2-b89746af7b8c None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:42.378869 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 313/618] 10.222.0.26 () {56 vars in 1149 bytes} [Wed Nov 2 03:21:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:21:42.384178 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1414faff-339e-4b28-aa6a-e9c8c7316605 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:42.480601 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb056e1-6ea2-46b4-97c2-b89746af7b8c None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:42.492992 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1414faff-339e-4b28-aa6a-e9c8c7316605 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:42.535419 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18775a08-38b0-4dd6-9d0e-9811dc339bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:42.536181 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18775a08-38b0-4dd6-9d0e-9811dc339bab None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:42.536525 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18775a08-38b0-4dd6-9d0e-9811dc339bab None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:42.536875 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18775a08-38b0-4dd6-9d0e-9811dc339bab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:42.556340 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:42.650397 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb056e1-6ea2-46b4-97c2-b89746af7b8c None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:42.667841 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18775a08-38b0-4dd6-9d0e-9811dc339bab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:42.672244 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18775a08-38b0-4dd6-9d0e-9811dc339bab None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:42.730240 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c39476-b43a-477f-9702-733a2818e503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:42.730880 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c39476-b43a-477f-9702-733a2818e503 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:42.731518 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c39476-b43a-477f-9702-733a2818e503 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:42.731518 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c39476-b43a-477f-9702-733a2818e503 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:42.732008 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c39476-b43a-477f-9702-733a2818e503 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:42.733365 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c39476-b43a-477f-9702-733a2818e503 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:42.742773 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-d7c39476-b43a-477f-9702-733a2818e503 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 03:21:42.742773 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:42.742773 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 03:21:42.742773 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 03:21:42.742773 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 03:21:42.742773 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 03:21:42.742773 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 03:21:42.742773 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 03:21:42.742773 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 03:21:42.742773 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:42.742773 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 03:21:42.742773 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:42.742773 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:42.742773 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:42.742773 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:42.742773 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:42.742773 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:42.743794 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:42.743794 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:42.743794 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:42.743794 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:42.743794 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:42.743794 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:42.743794 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 03:21:42.743794 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 03:21:42.743794 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 03:21:42.743794 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 03:21:42.743794 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:42.743794 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:42.743794 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:42.743794 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:42.743794 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:42.743794 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:42.745371 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:42.745371 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:42.745371 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:42.745371 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:42.745371 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:42.745371 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:42.745371 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:42.745371 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:42.745371 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:42.745371 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:42.745371 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 03:21:42.745371 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 03:21:42.745371 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Nov 02 03:21:42.745371 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:42.745371 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 314/619] 10.222.0.26 () {56 vars in 1129 bytes} [Wed Nov 2 03:21:42 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 03:21:42.749867 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c39476-b43a-477f-9702-733a2818e503 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:42.780361 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:42.790232 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 306/620] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Nov 2 03:21:42 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Nov 02 03:21:42.800215 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18775a08-38b0-4dd6-9d0e-9811dc339bab None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:42.863584 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c39476-b43a-477f-9702-733a2818e503 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:42.869125 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:42.903305 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ed4760-66e3-4fd3-aaf7-9a52f52c8523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:42.904454 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed4760-66e3-4fd3-aaf7-9a52f52c8523 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:42.905104 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed4760-66e3-4fd3-aaf7-9a52f52c8523 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:42.906169 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed4760-66e3-4fd3-aaf7-9a52f52c8523 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:42.907572 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ed4760-66e3-4fd3-aaf7-9a52f52c8523 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:42.908278 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ed4760-66e3-4fd3-aaf7-9a52f52c8523 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:42.910045 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ed4760-66e3-4fd3-aaf7-9a52f52c8523 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:42.915670 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 315/621] 10.222.0.26 () {56 vars in 1124 bytes} [Wed Nov 2 03:21:42 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:42.921481 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ed4760-66e3-4fd3-aaf7-9a52f52c8523 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:43.002686 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18775a08-38b0-4dd6-9d0e-9811dc339bab None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:43.051160 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:43.064173 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ed4760-66e3-4fd3-aaf7-9a52f52c8523 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:43.099590 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b02df9-d51d-439a-8a86-bbd2cb806258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:43.100593 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b02df9-d51d-439a-8a86-bbd2cb806258 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:43.101192 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b02df9-d51d-439a-8a86-bbd2cb806258 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:43.102341 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b02df9-d51d-439a-8a86-bbd2cb806258 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:43.113849 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a606f847-4afb-4348-a047-4f6bd676d1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:43.114696 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606f847-4afb-4348-a047-4f6bd676d1bc None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:43.115125 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606f847-4afb-4348-a047-4f6bd676d1bc None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:43.115536 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a606f847-4afb-4348-a047-4f6bd676d1bc None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:43.116849 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a606f847-4afb-4348-a047-4f6bd676d1bc None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:43.119086 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a606f847-4afb-4348-a047-4f6bd676d1bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:43.120617 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a606f847-4afb-4348-a047-4f6bd676d1bc None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:43.123668 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b02df9-d51d-439a-8a86-bbd2cb806258 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:43.125933 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b02df9-d51d-439a-8a86-bbd2cb806258 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:43.137477 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 316/622] 10.222.0.26 () {56 vars in 1124 bytes} [Wed Nov 2 03:21:42 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:43.151813 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-e7b02df9-d51d-439a-8a86-bbd2cb806258 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:43.151813 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:43.151813 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:43.151813 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:43.151813 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:43.151813 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:43.151813 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:43.151813 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:43.151813 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:43.151813 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:43.151813 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:43.151813 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:43.151813 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 03:21:43.151813 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 03:21:43.151813 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 03:21:43.151813 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 03:21:43.153003 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:43.153003 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:43.153003 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:43.153003 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:43.153003 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:43.153003 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:43.153003 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:43.153003 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:43.153003 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:43.153003 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:43.153003 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:43.153003 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:43.153003 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:43.153003 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:43.153003 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:43.153003 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:43.153003 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 03:21:43.153987 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 03:21:43.153987 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:43.153987 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:43.155398 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a606f847-4afb-4348-a047-4f6bd676d1bc None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:43.166936 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 307/623] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:21:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 03:21:43.173629 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b02df9-d51d-439a-8a86-bbd2cb806258 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:43.288831 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:43.300419 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b02df9-d51d-439a-8a86-bbd2cb806258 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:43.343108 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e21de28-b735-4279-93a6-7d35f8d7de6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:43.344400 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e21de28-b735-4279-93a6-7d35f8d7de6c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:43.345251 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e21de28-b735-4279-93a6-7d35f8d7de6c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:43.345672 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e21de28-b735-4279-93a6-7d35f8d7de6c None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:43.346461 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e21de28-b735-4279-93a6-7d35f8d7de6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:43.346905 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e21de28-b735-4279-93a6-7d35f8d7de6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:43.348996 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e21de28-b735-4279-93a6-7d35f8d7de6c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:43.362798 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 308/624] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Nov 2 03:21:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:43.368871 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e21de28-b735-4279-93a6-7d35f8d7de6c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:43.379836 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a606f847-4afb-4348-a047-4f6bd676d1bc None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:43.445446 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b838cf-3ca8-4683-8f0e-4e1dcc10fa37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:43.446518 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b838cf-3ca8-4683-8f0e-4e1dcc10fa37 None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:43.446640 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b838cf-3ca8-4683-8f0e-4e1dcc10fa37 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:43.447033 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b838cf-3ca8-4683-8f0e-4e1dcc10fa37 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:43.447805 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b838cf-3ca8-4683-8f0e-4e1dcc10fa37 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:43.449308 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b838cf-3ca8-4683-8f0e-4e1dcc10fa37 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:43.480035 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 317/625] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:21:43 2022] POST /identity/v3/endpoints => generated 346 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 03:21:43.480035 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317eb843-8f8a-4286-bf9b-1079d149ea36 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:43.480035 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317eb843-8f8a-4286-bf9b-1079d149ea36 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:43.480035 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317eb843-8f8a-4286-bf9b-1079d149ea36 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:43.486201 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 318/626] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:21:43.488461 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c921317-6d2b-4ae5-8e5a-a6bf8ba9c8e6 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:43.488860 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c921317-6d2b-4ae5-8e5a-a6bf8ba9c8e6 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:43.489168 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e21de28-b735-4279-93a6-7d35f8d7de6c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:43.489621 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c921317-6d2b-4ae5-8e5a-a6bf8ba9c8e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:43.521958 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-9c921317-6d2b-4ae5-8e5a-a6bf8ba9c8e6 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:43.524664 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-9c921317-6d2b-4ae5-8e5a-a6bf8ba9c8e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:43.542175 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:43.588700 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e35ce-34e1-4c85-9e97-672884087cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:43.589625 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e35ce-34e1-4c85-9e97-672884087cfe None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:43.589911 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e35ce-34e1-4c85-9e97-672884087cfe None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:43.590182 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44e35ce-34e1-4c85-9e97-672884087cfe None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:43.606689 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44e35ce-34e1-4c85-9e97-672884087cfe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:43.611727 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44e35ce-34e1-4c85-9e97-672884087cfe None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:43.641640 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-a44e35ce-34e1-4c85-9e97-672884087cfe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 03:21:43.641640 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:43.641640 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:43.641640 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:43.641640 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:43.641640 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:43.641640 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:43.641640 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:43.641640 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:43.641640 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:43.641640 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:43.641640 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:43.641640 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 03:21:43.641640 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 03:21:43.641640 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 03:21:43.642592 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 03:21:43.642592 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:43.642592 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:43.642592 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:43.642592 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:43.642592 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:43.642592 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:43.642592 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:43.642592 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:43.642592 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:43.642592 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:43.642592 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:43.642592 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:43.642592 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:43.642592 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:43.642592 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:43.642592 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:43.643477 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 03:21:43.643477 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 03:21:43.643477 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 03:21:43.643477 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 03:21:43.643477 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 03:21:43.643477 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 03:21:43.643477 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 03:21:43.643477 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:43.644829 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 309/627] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Nov 2 03:21:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 03:21:43.651559 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e35ce-34e1-4c85-9e97-672884087cfe None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:43.682713 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:43.717998 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 319/628] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 03:21:43.786891 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44e35ce-34e1-4c85-9e97-672884087cfe None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:43.822811 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216a7246-ba96-4ce3-a4fa-ac4f04a1276a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:43.823430 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216a7246-ba96-4ce3-a4fa-ac4f04a1276a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:43.823708 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216a7246-ba96-4ce3-a4fa-ac4f04a1276a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:43.824005 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216a7246-ba96-4ce3-a4fa-ac4f04a1276a None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:43.824703 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216a7246-ba96-4ce3-a4fa-ac4f04a1276a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:43.825006 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216a7246-ba96-4ce3-a4fa-ac4f04a1276a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:43.826443 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216a7246-ba96-4ce3-a4fa-ac4f04a1276a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:43.841533 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 310/629] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Nov 2 03:21:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:43.854936 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c921317-6d2b-4ae5-8e5a-a6bf8ba9c8e6 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:43.978169 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c921317-6d2b-4ae5-8e5a-a6bf8ba9c8e6 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:44.017459 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5366f1e4-51d0-4183-ad2d-09e2bc345889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:44.018535 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5366f1e4-51d0-4183-ad2d-09e2bc345889 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:44.019189 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5366f1e4-51d0-4183-ad2d-09e2bc345889 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:44.019699 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5366f1e4-51d0-4183-ad2d-09e2bc345889 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:44.020729 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5366f1e4-51d0-4183-ad2d-09e2bc345889 None admin] RBAC: Adding query filter params (role.id=e2a148d6fd534007bbb44d586d4306d1, scope.project.id=d0b5e7017ac64ae6ab62c1a05aa54b72, user.id=da7ee607b66e43f3823d9d2faf373974) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:44.021311 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5366f1e4-51d0-4183-ad2d-09e2bc345889 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:44.023008 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5366f1e4-51d0-4183-ad2d-09e2bc345889 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:44.036430 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 320/630] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Nov 2 03:21:43 2022] GET /identity/v3/role_assignments?user.id=da7ee607b66e43f3823d9d2faf373974&scope.project.id=d0b5e7017ac64ae6ab62c1a05aa54b72&role.id=e2a148d6fd534007bbb44d586d4306d1 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:44.049132 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216a7246-ba96-4ce3-a4fa-ac4f04a1276a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:44.224867 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:44.279593 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216a7246-ba96-4ce3-a4fa-ac4f04a1276a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:44.330520 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcb93c5-7604-456f-b005-2ab37e0a0ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:44.331161 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb93c5-7604-456f-b005-2ab37e0a0ab6 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:44.331440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb93c5-7604-456f-b005-2ab37e0a0ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:44.331716 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcb93c5-7604-456f-b005-2ab37e0a0ab6 None admin] PATH_INFO: `/v3/services/network` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:44.332331 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcb93c5-7604-456f-b005-2ab37e0a0ab6 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:44.334075 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcb93c5-7604-456f-b005-2ab37e0a0ab6 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:44.343692 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-afcb93c5-7604-456f-b005-2ab37e0a0ab6 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 03:21:44.343692 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:44.343692 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 03:21:44.343692 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 03:21:44.343692 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 03:21:44.343692 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 03:21:44.343692 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 03:21:44.343692 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 03:21:44.343692 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 03:21:44.343692 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:44.343692 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 03:21:44.343692 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:44.343692 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:44.343692 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:44.343692 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:44.343692 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:44.343692 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:44.344748 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:44.344748 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:44.344748 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:44.344748 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:44.344748 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:44.344748 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:44.344748 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 03:21:44.344748 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 03:21:44.344748 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 03:21:44.344748 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 03:21:44.344748 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:44.344748 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:44.344748 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:44.344748 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:44.344748 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:44.344748 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:44.345571 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:44.345571 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:44.345571 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:44.345571 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:44.345571 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:44.345571 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:44.345571 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:44.345571 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:44.345571 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:44.345571 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:44.345571 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 03:21:44.345571 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 03:21:44.345571 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Nov 02 03:21:44.345571 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:44.350689 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 311/631] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Nov 2 03:21:44 2022] GET /identity/v3/services/network => generated 88 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:44.351715 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5366f1e4-51d0-4183-ad2d-09e2bc345889 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:44.356701 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:44.411098 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919b634-efd7-4342-bd67-5ce3913722b2 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:44.411417 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919b634-efd7-4342-bd67-5ce3913722b2 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:44.411645 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d919b634-efd7-4342-bd67-5ce3913722b2 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:44.413464 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 312/632] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 03:21:44.421570 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2472f4fb-a0f9-47a4-9537-a763a6d74006 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:44.421988 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2472f4fb-a0f9-47a4-9537-a763a6d74006 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:44.422369 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2472f4fb-a0f9-47a4-9537-a763a6d74006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:44.447698 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-2472f4fb-a0f9-47a4-9537-a763a6d74006 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:44.449085 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2472f4fb-a0f9-47a4-9537-a763a6d74006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:44.513973 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5366f1e4-51d0-4183-ad2d-09e2bc345889 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:44.583455 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f374a9-6ade-4800-a56f-2bd2402f6b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:44.584006 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f374a9-6ade-4800-a56f-2bd2402f6b17 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:44.584290 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f374a9-6ade-4800-a56f-2bd2402f6b17 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:44.585095 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f374a9-6ade-4800-a56f-2bd2402f6b17 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:44.585733 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f374a9-6ade-4800-a56f-2bd2402f6b17 None admin] RBAC: Adding query filter params (name=network) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:44.586036 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f374a9-6ade-4800-a56f-2bd2402f6b17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:44.587400 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f374a9-6ade-4800-a56f-2bd2402f6b17 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:44.590543 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 313/633] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 03:21:44.594533 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 321/634] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Nov 2 03:21:44 2022] GET /identity/v3/services?name=network => generated 124 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:44.599915 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2472f4fb-a0f9-47a4-9537-a763a6d74006 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:44.694297 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:44.754149 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2472f4fb-a0f9-47a4-9537-a763a6d74006 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:44.799038 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074f62f2-49fb-45d3-9642-b14245e53d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:44.800029 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074f62f2-49fb-45d3-9642-b14245e53d21 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:44.800506 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074f62f2-49fb-45d3-9642-b14245e53d21 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:44.800972 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074f62f2-49fb-45d3-9642-b14245e53d21 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:44.801812 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074f62f2-49fb-45d3-9642-b14245e53d21 None admin] RBAC: Adding query filter params (type=network) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:44.802335 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074f62f2-49fb-45d3-9642-b14245e53d21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:44.804012 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074f62f2-49fb-45d3-9642-b14245e53d21 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:44.809700 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 314/635] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Nov 2 03:21:44 2022] GET /identity/v3/services?type=network => generated 350 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:44.815083 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f374a9-6ade-4800-a56f-2bd2402f6b17 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:44.851530 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2be3a8-f461-4304-893d-57fcf039a665 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:44.852179 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2be3a8-f461-4304-893d-57fcf039a665 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:44.852651 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2be3a8-f461-4304-893d-57fcf039a665 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:44.854404 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 315/636] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 03:21:44.859369 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cda219-ce22-4275-8434-4eca1671efca None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:44.859868 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cda219-ce22-4275-8434-4eca1671efca None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:44.860389 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cda219-ce22-4275-8434-4eca1671efca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:44.879635 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-e1cda219-ce22-4275-8434-4eca1671efca None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:44.881163 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e1cda219-ce22-4275-8434-4eca1671efca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:44.942436 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:44.962364 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 316/637] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 03:21:44.982206 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cda219-ce22-4275-8434-4eca1671efca None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:45.014550 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f374a9-6ade-4800-a56f-2bd2402f6b17 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:45.090482 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df05d75a-6773-4d66-9f26-909e90b06bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:45.091098 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05d75a-6773-4d66-9f26-909e90b06bc8 None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:45.091379 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05d75a-6773-4d66-9f26-909e90b06bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:45.091650 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df05d75a-6773-4d66-9f26-909e90b06bc8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:45.092282 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:45.092458 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df05d75a-6773-4d66-9f26-909e90b06bc8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:45.094571 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df05d75a-6773-4d66-9f26-909e90b06bc8 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:45.127537 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 322/638] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:21:44 2022] POST /identity/v3/endpoints => generated 337 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:21:45.165276 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cda219-ce22-4275-8434-4eca1671efca None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:45.215982 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7010db75-2433-489b-a758-806fcc85702c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:45.216666 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7010db75-2433-489b-a758-806fcc85702c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:45.216948 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7010db75-2433-489b-a758-806fcc85702c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:45.217224 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7010db75-2433-489b-a758-806fcc85702c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:45.217821 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7010db75-2433-489b-a758-806fcc85702c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:45.219319 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7010db75-2433-489b-a758-806fcc85702c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:45.233345 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-7010db75-2433-489b-a758-806fcc85702c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 03:21:45.233345 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:45.233345 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 03:21:45.233345 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 03:21:45.233345 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 03:21:45.233345 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 03:21:45.233345 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 03:21:45.233345 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 03:21:45.233345 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 03:21:45.233345 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:45.233345 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 03:21:45.233345 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:45.233345 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:45.233345 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:45.233345 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:45.233345 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:45.233345 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:45.234447 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:45.234447 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:45.234447 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:45.234447 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:45.234447 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:45.234447 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:45.234447 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 03:21:45.234447 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 03:21:45.234447 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 03:21:45.234447 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 03:21:45.234447 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:45.234447 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:45.234447 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:45.234447 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:45.234447 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:45.234447 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:45.235270 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:45.235270 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:45.235270 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:45.235270 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:45.235270 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:45.235270 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:45.235270 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:45.235270 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:45.235270 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:45.235270 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:45.235270 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 03:21:45.235270 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 03:21:45.235270 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 03:21:45.235270 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:45.235270 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 317/639] 10.222.0.26 () {56 vars in 1126 bytes} [Wed Nov 2 03:21:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:45.240996 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df05d75a-6773-4d66-9f26-909e90b06bc8 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:45.255788 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7010db75-2433-489b-a758-806fcc85702c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:45.371600 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7010db75-2433-489b-a758-806fcc85702c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:45.390959 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:45.420548 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b78b78-aff6-4dbf-9453-610b422cb595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:45.421166 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b78b78-aff6-4dbf-9453-610b422cb595 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:45.421444 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b78b78-aff6-4dbf-9453-610b422cb595 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:45.421716 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b78b78-aff6-4dbf-9453-610b422cb595 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:45.422381 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b78b78-aff6-4dbf-9453-610b422cb595 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:45.423727 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b78b78-aff6-4dbf-9453-610b422cb595 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:45.461671 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 318/640] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Nov 2 03:21:45 2022] POST /identity/v3/services => generated 269 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 03:21:45.486372 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df05d75a-6773-4d66-9f26-909e90b06bc8 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:45.517805 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:45.540173 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f1876d-ae98-42af-a923-20e8f13526d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:45.540173 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f1876d-ae98-42af-a923-20e8f13526d4 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:45.540173 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f1876d-ae98-42af-a923-20e8f13526d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:45.540173 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f1876d-ae98-42af-a923-20e8f13526d4 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:45.540574 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f1876d-ae98-42af-a923-20e8f13526d4 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:45.540899 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f1876d-ae98-42af-a923-20e8f13526d4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:45.543142 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f1876d-ae98-42af-a923-20e8f13526d4 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:45.549034 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 323/641] 10.222.0.26 () {56 vars in 1122 bytes} [Wed Nov 2 03:21:45 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:45.554526 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b78b78-aff6-4dbf-9453-610b422cb595 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:45.691996 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b78b78-aff6-4dbf-9453-610b422cb595 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:45.728303 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f329cc4-b83d-47cc-bb94-1b6453f6e41c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:45.729016 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f329cc4-b83d-47cc-bb94-1b6453f6e41c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:45.729293 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f329cc4-b83d-47cc-bb94-1b6453f6e41c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:45.729563 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f329cc4-b83d-47cc-bb94-1b6453f6e41c None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:45.730192 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f329cc4-b83d-47cc-bb94-1b6453f6e41c None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:45.730491 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f329cc4-b83d-47cc-bb94-1b6453f6e41c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:45.731858 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f329cc4-b83d-47cc-bb94-1b6453f6e41c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:45.736912 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 319/642] 10.222.0.26 () {56 vars in 1122 bytes} [Wed Nov 2 03:21:45 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:46.256518 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b0a01f-43bb-47c7-adcd-c502459672dd None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:46.257062 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b0a01f-43bb-47c7-adcd-c502459672dd None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:46.257062 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b0a01f-43bb-47c7-adcd-c502459672dd None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:46.258685 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 324/643] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:21:46.264010 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e889fc6-2f39-420d-af04-c7fcdbfb866d None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:46.264304 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e889fc6-2f39-420d-af04-c7fcdbfb866d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:46.264586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e889fc6-2f39-420d-af04-c7fcdbfb866d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:46.283239 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-1e889fc6-2f39-420d-af04-c7fcdbfb866d None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:46.284469 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1e889fc6-2f39-420d-af04-c7fcdbfb866d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:46.320878 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:46.357467 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 320/644] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:46 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 03:21:46.622476 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b0a01f-43bb-47c7-adcd-c502459672dd None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:46.731870 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b0a01f-43bb-47c7-adcd-c502459672dd None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:46.762377 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279df119-5fe5-4e56-b94a-6edae43bbdd7 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:46.762897 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279df119-5fe5-4e56-b94a-6edae43bbdd7 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:46.763163 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279df119-5fe5-4e56-b94a-6edae43bbdd7 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:46.767977 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 321/645] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:21:46.772411 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db2e0ed-a8dc-4e90-af6b-17e66c487f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:46.772411 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db2e0ed-a8dc-4e90-af6b-17e66c487f57 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:46.772638 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08e84b6-f369-447e-9482-169868817406 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:46.772721 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db2e0ed-a8dc-4e90-af6b-17e66c487f57 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:46.772793 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08e84b6-f369-447e-9482-169868817406 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:46.772861 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db2e0ed-a8dc-4e90-af6b-17e66c487f57 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:46.772939 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08e84b6-f369-447e-9482-169868817406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:46.773018 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db2e0ed-a8dc-4e90-af6b-17e66c487f57 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:46.773018 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db2e0ed-a8dc-4e90-af6b-17e66c487f57 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:46.783767 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-5db2e0ed-a8dc-4e90-af6b-17e66c487f57 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 03:21:46.783767 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:46.783767 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 03:21:46.783767 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 03:21:46.783767 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 03:21:46.783767 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 03:21:46.783767 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 03:21:46.783767 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 03:21:46.783767 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 03:21:46.783767 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:46.783767 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 03:21:46.783767 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:46.783767 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:46.783767 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:46.783767 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:46.783767 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:46.783767 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:46.784804 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:46.784804 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:46.784804 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:46.784804 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:46.784804 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:46.784804 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:46.784804 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 03:21:46.784804 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 03:21:46.784804 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 03:21:46.784804 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 03:21:46.784804 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:46.784804 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:46.784804 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:46.784804 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:46.784804 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:46.784804 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:46.785679 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:46.785679 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:46.785679 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:46.785679 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:46.785679 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:46.785679 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:46.785679 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:46.785679 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:46.785679 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:46.785679 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:46.785679 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 03:21:46.785679 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 03:21:46.785679 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 03:21:46.785679 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:46.785679 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 325/646] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Nov 2 03:21:46 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:46.792458 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d08e84b6-f369-447e-9482-169868817406 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:46.793038 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db2e0ed-a8dc-4e90-af6b-17e66c487f57 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:46.793746 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d08e84b6-f369-447e-9482-169868817406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:46.849811 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:46.882507 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 322/647] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:46 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 03:21:46.906549 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db2e0ed-a8dc-4e90-af6b-17e66c487f57 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:46.943717 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61049918-4864-4b64-b88a-a93b20927341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:46.944658 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61049918-4864-4b64-b88a-a93b20927341 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:46.945156 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61049918-4864-4b64-b88a-a93b20927341 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:46.945607 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61049918-4864-4b64-b88a-a93b20927341 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:46.946406 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61049918-4864-4b64-b88a-a93b20927341 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:46.946886 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61049918-4864-4b64-b88a-a93b20927341 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:46.948456 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61049918-4864-4b64-b88a-a93b20927341 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:46.953669 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 326/648] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Nov 2 03:21:46 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:46.959986 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08e84b6-f369-447e-9482-169868817406 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:47.062074 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:47.065079 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08e84b6-f369-447e-9482-169868817406 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:47.100820 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3961e9-c638-4123-a5b1-18c0d092c407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:47.101439 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3961e9-c638-4123-a5b1-18c0d092c407 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:47.101777 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3961e9-c638-4123-a5b1-18c0d092c407 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:47.102074 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3961e9-c638-4123-a5b1-18c0d092c407 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:47.102765 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3961e9-c638-4123-a5b1-18c0d092c407 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:47.103077 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3961e9-c638-4123-a5b1-18c0d092c407 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:47.104562 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3961e9-c638-4123-a5b1-18c0d092c407 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:47.109431 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 323/649] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Nov 2 03:21:46 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:47.148693 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61049918-4864-4b64-b88a-a93b20927341 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:47.262091 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61049918-4864-4b64-b88a-a93b20927341 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:47.276978 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:47.302232 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90a77d-77a7-46df-b11a-060ab472c4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:47.302878 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90a77d-77a7-46df-b11a-060ab472c4b7 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:47.303221 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90a77d-77a7-46df-b11a-060ab472c4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:47.303504 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b90a77d-77a7-46df-b11a-060ab472c4b7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:47.315322 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b90a77d-77a7-46df-b11a-060ab472c4b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:21:47.320926 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-4b90a77d-77a7-46df-b11a-060ab472c4b7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 03:21:47.320926 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:47.320926 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:47.320926 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:47.320926 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:47.320926 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:47.320926 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:47.320926 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:47.320926 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:47.320926 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:47.320926 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:47.320926 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:47.320926 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:21:47.320926 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:21:47.320926 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:21:47.320926 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:21:47.321854 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:21:47.321854 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:21:47.321854 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:21:47.321854 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:21:47.321854 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:47.321854 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:47.321854 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:47.321854 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:47.321854 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:47.321854 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:47.321854 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:47.321854 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:47.321854 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:47.321854 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:47.321854 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:47.321854 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:47.322699 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:47.322699 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:47.322699 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:47.322699 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:47.322699 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:21:47.322699 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:21:47.322699 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:21:47.322699 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:21:47.322699 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:21:47.322699 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:21:47.322699 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 03:21:47.322699 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:47.326093 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 327/650] 10.222.0.26 () {56 vars in 1120 bytes} [Wed Nov 2 03:21:47 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:47.329074 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3961e9-c638-4123-a5b1-18c0d092c407 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:47.436175 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3961e9-c638-4123-a5b1-18c0d092c407 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:47.471262 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2790ab7d-f3fd-46e2-bb88-87c904c0d920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:47.472235 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790ab7d-f3fd-46e2-bb88-87c904c0d920 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:47.472689 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790ab7d-f3fd-46e2-bb88-87c904c0d920 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:47.473127 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2790ab7d-f3fd-46e2-bb88-87c904c0d920 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:47.473956 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2790ab7d-f3fd-46e2-bb88-87c904c0d920 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:47.474448 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2790ab7d-f3fd-46e2-bb88-87c904c0d920 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:47.476074 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2790ab7d-f3fd-46e2-bb88-87c904c0d920 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:47.489808 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 324/651] 10.222.0.26 () {56 vars in 1115 bytes} [Wed Nov 2 03:21:47 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:47.491971 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90a77d-77a7-46df-b11a-060ab472c4b7 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:47.604452 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b90a77d-77a7-46df-b11a-060ab472c4b7 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:47.644880 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f55085-32ac-4b4b-ad6c-19b0d3afd154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:47.645544 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f55085-32ac-4b4b-ad6c-19b0d3afd154 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:47.645823 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f55085-32ac-4b4b-ad6c-19b0d3afd154 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:47.646098 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f55085-32ac-4b4b-ad6c-19b0d3afd154 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:47.658791 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f55085-32ac-4b4b-ad6c-19b0d3afd154 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:47.660276 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f55085-32ac-4b4b-ad6c-19b0d3afd154 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:47.676346 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-60f55085-32ac-4b4b-ad6c-19b0d3afd154 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:47.676346 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:47.676346 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:47.676346 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:47.676346 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:47.676346 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:47.676346 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:47.676346 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:47.676346 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:47.676346 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:47.676346 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:47.676346 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:47.676346 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 03:21:47.676346 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 03:21:47.676346 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 03:21:47.676346 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 03:21:47.677362 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:47.677362 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:47.677362 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:47.677362 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:47.677362 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:47.677362 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:47.677362 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:47.677362 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:47.677362 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:47.677362 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:47.677362 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:47.677362 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:47.677362 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:47.677362 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:47.677362 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:47.677362 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:47.677362 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 03:21:47.679123 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 03:21:47.679123 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:47.679123 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:47.679123 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 328/652] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:21:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:47.685391 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2790ab7d-f3fd-46e2-bb88-87c904c0d920 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:47.717910 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:47.748528 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c102a99-1489-428f-b60c-9bbd76223453 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:47.749164 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c102a99-1489-428f-b60c-9bbd76223453 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:47.749633 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c102a99-1489-428f-b60c-9bbd76223453 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:47.751503 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 329/653] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 03:21:47.756093 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b16d6-2ba7-4fd4-9d10-ed956d21a429 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:47.756661 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b16d6-2ba7-4fd4-9d10-ed956d21a429 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:47.757130 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597b16d6-2ba7-4fd4-9d10-ed956d21a429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:47.778391 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-597b16d6-2ba7-4fd4-9d10-ed956d21a429 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:47.780488 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-597b16d6-2ba7-4fd4-9d10-ed956d21a429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:47.798340 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2790ab7d-f3fd-46e2-bb88-87c904c0d920 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:47.805444 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:47.835862 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dddea0f-e07d-49fe-9edf-aa52ece94afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:47.836891 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dddea0f-e07d-49fe-9edf-aa52ece94afb None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:47.837167 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dddea0f-e07d-49fe-9edf-aa52ece94afb None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:47.837430 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dddea0f-e07d-49fe-9edf-aa52ece94afb None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:47.838126 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dddea0f-e07d-49fe-9edf-aa52ece94afb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:47.838421 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dddea0f-e07d-49fe-9edf-aa52ece94afb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:47.839823 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dddea0f-e07d-49fe-9edf-aa52ece94afb None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:47.857852 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 325/654] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Nov 2 03:21:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:47.859367 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dddea0f-e07d-49fe-9edf-aa52ece94afb None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:47.876906 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 330/655] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:47 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 03:21:47.960246 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:47.978919 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dddea0f-e07d-49fe-9edf-aa52ece94afb None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:48.008200 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df2e901-512c-443e-99a5-9f024195ee94 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:48.008523 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df2e901-512c-443e-99a5-9f024195ee94 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:48.008760 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df2e901-512c-443e-99a5-9f024195ee94 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:48.010466 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 331/656] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 03:21:48.015416 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc86e0b-9d44-495b-a1ab-c49be9816087 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:48.015700 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc86e0b-9d44-495b-a1ab-c49be9816087 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:48.015996 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc86e0b-9d44-495b-a1ab-c49be9816087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:48.018175 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671bb5e2-722e-4e69-a278-7d7ea8db80d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:48.019223 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671bb5e2-722e-4e69-a278-7d7ea8db80d5 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:48.019781 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671bb5e2-722e-4e69-a278-7d7ea8db80d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:48.020262 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671bb5e2-722e-4e69-a278-7d7ea8db80d5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:48.029876 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671bb5e2-722e-4e69-a278-7d7ea8db80d5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:48.031662 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671bb5e2-722e-4e69-a278-7d7ea8db80d5 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:48.034073 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-0dc86e0b-9d44-495b-a1ab-c49be9816087 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:48.035334 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-0dc86e0b-9d44-495b-a1ab-c49be9816087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:48.044778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-671bb5e2-722e-4e69-a278-7d7ea8db80d5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 03:21:48.044778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:48.044778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:48.044778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:48.044778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:48.044778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:48.044778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:48.044778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:48.044778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:48.044778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:48.044778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:48.044778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:48.044778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 03:21:48.044778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 03:21:48.044778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 03:21:48.044778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 03:21:48.045715 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:48.045715 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:48.045715 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 03:21:48.045715 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 03:21:48.045715 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 03:21:48.045715 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 03:21:48.045715 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:48.045715 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:48.045715 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:48.045715 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:48.045715 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:48.045715 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:48.045715 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:48.045715 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:48.045715 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:48.045715 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:48.045715 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:48.045715 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:48.046656 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:48.046656 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:48.046656 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 03:21:48.046656 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 03:21:48.046656 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 03:21:48.046656 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 03:21:48.046656 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 03:21:48.046656 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 03:21:48.046656 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 03:21:48.046656 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:48.053321 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 326/657] 10.222.0.26 () {56 vars in 1117 bytes} [Wed Nov 2 03:21:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 03:21:48.055132 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671bb5e2-722e-4e69-a278-7d7ea8db80d5 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:48.120188 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 332/658] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:48 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 03:21:48.146149 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc86e0b-9d44-495b-a1ab-c49be9816087 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:48.176281 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671bb5e2-722e-4e69-a278-7d7ea8db80d5 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:48.214963 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea22625-676e-493d-9d73-52c228f6635c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:48.215576 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea22625-676e-493d-9d73-52c228f6635c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:48.215853 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea22625-676e-493d-9d73-52c228f6635c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:48.216170 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea22625-676e-493d-9d73-52c228f6635c None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:48.216923 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea22625-676e-493d-9d73-52c228f6635c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:48.217271 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea22625-676e-493d-9d73-52c228f6635c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:48.218853 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea22625-676e-493d-9d73-52c228f6635c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:48.242913 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 327/659] 10.222.0.26 () {56 vars in 1149 bytes} [Wed Nov 2 03:21:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:21:48.244901 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea22625-676e-493d-9d73-52c228f6635c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:48.249617 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:48.254063 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:48.272383 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc86e0b-9d44-495b-a1ab-c49be9816087 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:48.315750 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a2e2e9-2f75-4400-aa9b-74c2b7b1dca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:48.317183 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a2e2e9-2f75-4400-aa9b-74c2b7b1dca6 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:48.317927 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a2e2e9-2f75-4400-aa9b-74c2b7b1dca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:48.318398 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a2e2e9-2f75-4400-aa9b-74c2b7b1dca6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:48.319987 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a2e2e9-2f75-4400-aa9b-74c2b7b1dca6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:48.322038 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a2e2e9-2f75-4400-aa9b-74c2b7b1dca6 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:48.333673 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-b8a2e2e9-2f75-4400-aa9b-74c2b7b1dca6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 03:21:48.333673 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:48.333673 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 03:21:48.333673 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 03:21:48.333673 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 03:21:48.333673 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 03:21:48.333673 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 03:21:48.333673 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 03:21:48.333673 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 03:21:48.333673 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:48.333673 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 03:21:48.333673 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:48.333673 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:48.333673 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:48.333673 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:48.333673 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:48.334819 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:48.334819 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:48.334819 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:48.334819 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:48.334819 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:48.334819 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:48.334819 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:48.334819 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 03:21:48.334819 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 03:21:48.334819 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 03:21:48.334819 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 03:21:48.334819 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:48.334819 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:48.334819 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:48.334819 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:48.334819 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:48.334819 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:48.335704 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:48.335704 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:48.335704 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:48.335704 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:48.335704 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:48.335704 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:48.335704 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:48.335704 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:48.335704 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:48.335704 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:48.335704 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 03:21:48.335704 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 03:21:48.335704 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 03:21:48.335704 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:48.336515 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 333/660] 10.222.0.26 () {56 vars in 1132 bytes} [Wed Nov 2 03:21:48 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 03:21:48.342022 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a2e2e9-2f75-4400-aa9b-74c2b7b1dca6 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:48.366922 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea22625-676e-493d-9d73-52c228f6635c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:48.402349 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cbdcff-6aae-4c21-ab7d-6ac5ff8961c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:48.403155 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cbdcff-6aae-4c21-ab7d-6ac5ff8961c2 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:48.403452 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cbdcff-6aae-4c21-ab7d-6ac5ff8961c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:48.403722 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cbdcff-6aae-4c21-ab7d-6ac5ff8961c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:48.414997 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cbdcff-6aae-4c21-ab7d-6ac5ff8961c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:48.416592 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cbdcff-6aae-4c21-ab7d-6ac5ff8961c2 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:48.431797 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-90cbdcff-6aae-4c21-ab7d-6ac5ff8961c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:48.431797 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:48.431797 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:48.431797 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:48.431797 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:48.431797 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:48.431797 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:48.431797 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:48.431797 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:48.431797 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:48.431797 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:48.431797 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:48.431797 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 03:21:48.431797 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 03:21:48.431797 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 03:21:48.431797 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 03:21:48.432993 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:48.432993 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:48.432993 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:48.432993 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:48.432993 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:48.432993 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:48.432993 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:48.432993 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:48.432993 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:48.432993 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:48.432993 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:48.432993 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:48.432993 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:48.432993 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:48.432993 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:48.432993 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:48.432993 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 03:21:48.434886 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 03:21:48.434886 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:48.434886 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:48.434886 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 328/661] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:21:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 03:21:48.434886 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cbdcff-6aae-4c21-ab7d-6ac5ff8961c2 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:48.466591 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a2e2e9-2f75-4400-aa9b-74c2b7b1dca6 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:48.501438 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caa01fa-14de-4a32-b881-c25534811de3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:48.502178 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa01fa-14de-4a32-b881-c25534811de3 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:48.502458 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa01fa-14de-4a32-b881-c25534811de3 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:48.502727 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caa01fa-14de-4a32-b881-c25534811de3 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:48.503354 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caa01fa-14de-4a32-b881-c25534811de3 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:48.503654 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caa01fa-14de-4a32-b881-c25534811de3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:48.505188 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4caa01fa-14de-4a32-b881-c25534811de3 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:48.508681 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:48.514769 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 334/662] 10.222.0.26 () {56 vars in 1126 bytes} [Wed Nov 2 03:21:48 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:48.515648 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caa01fa-14de-4a32-b881-c25534811de3 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:48.539781 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:48.559096 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cbdcff-6aae-4c21-ab7d-6ac5ff8961c2 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:48.598116 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272d4cd6-4606-440e-9267-47ed89149e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:48.598725 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d4cd6-4606-440e-9267-47ed89149e09 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:48.599009 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d4cd6-4606-440e-9267-47ed89149e09 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:48.599280 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d4cd6-4606-440e-9267-47ed89149e09 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:48.599926 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272d4cd6-4606-440e-9267-47ed89149e09 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:48.601877 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272d4cd6-4606-440e-9267-47ed89149e09 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:48.617855 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 329/663] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Nov 2 03:21:48 2022] POST /identity/v3/services => generated 241 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:21:48.619080 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272d4cd6-4606-440e-9267-47ed89149e09 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:48.635348 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4caa01fa-14de-4a32-b881-c25534811de3 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:48.703098 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917ec6d0-592f-4a76-ab36-1119dca3d671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:48.704420 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ec6d0-592f-4a76-ab36-1119dca3d671 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:48.704926 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ec6d0-592f-4a76-ab36-1119dca3d671 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:48.705417 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917ec6d0-592f-4a76-ab36-1119dca3d671 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:48.706583 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917ec6d0-592f-4a76-ab36-1119dca3d671 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:48.707127 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917ec6d0-592f-4a76-ab36-1119dca3d671 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:48.709908 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917ec6d0-592f-4a76-ab36-1119dca3d671 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:48.729375 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 335/664] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Nov 2 03:21:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:48.737241 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917ec6d0-592f-4a76-ab36-1119dca3d671 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:48.752005 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272d4cd6-4606-440e-9267-47ed89149e09 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:48.774330 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:48.798379 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e96c77-3b7b-4046-a337-706de5c0ed40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:48.799716 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e96c77-3b7b-4046-a337-706de5c0ed40 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:48.800322 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e96c77-3b7b-4046-a337-706de5c0ed40 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:48.800593 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e96c77-3b7b-4046-a337-706de5c0ed40 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:48.801222 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e96c77-3b7b-4046-a337-706de5c0ed40 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:48.801520 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e96c77-3b7b-4046-a337-706de5c0ed40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:48.802865 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e96c77-3b7b-4046-a337-706de5c0ed40 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:48.810392 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 330/665] 10.222.0.26 () {56 vars in 1126 bytes} [Wed Nov 2 03:21:48 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:48.815441 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:48.817793 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e96c77-3b7b-4046-a337-706de5c0ed40 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:48.865064 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917ec6d0-592f-4a76-ab36-1119dca3d671 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:48.911988 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aee6566-830a-4df0-84f8-495f3e6821df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:48.913603 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee6566-830a-4df0-84f8-495f3e6821df None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:48.913885 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee6566-830a-4df0-84f8-495f3e6821df None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:48.914422 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aee6566-830a-4df0-84f8-495f3e6821df None admin] PATH_INFO: `/v3/projects/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:48.923637 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aee6566-830a-4df0-84f8-495f3e6821df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:48.925369 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aee6566-830a-4df0-84f8-495f3e6821df None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:48.936175 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e96c77-3b7b-4046-a337-706de5c0ed40 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:48.941005 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-9aee6566-830a-4df0-84f8-495f3e6821df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 03:21:48.941005 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:48.941005 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:48.941005 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:48.941005 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:48.941005 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:48.941005 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:48.941005 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:48.941005 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:48.941005 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:48.941005 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:48.941005 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:48.941005 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 03:21:48.941005 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 03:21:48.941005 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 03:21:48.941863 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 03:21:48.941863 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:48.941863 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:48.941863 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:48.941863 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:48.941863 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:48.941863 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:48.941863 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:48.941863 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:48.941863 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:48.941863 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:48.941863 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:48.941863 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:48.941863 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:48.941863 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:48.941863 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:48.941863 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:48.942758 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 03:21:48.942758 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 03:21:48.942758 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 03:21:48.942758 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 03:21:48.942758 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 03:21:48.942758 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 03:21:48.942758 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 03:21:48.942758 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:48.942758 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 336/666] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Nov 2 03:21:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:48.952982 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aee6566-830a-4df0-84f8-495f3e6821df None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:48.979251 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a02103-07b2-4947-9ce7-2506e0dbf037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:48.980400 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a02103-07b2-4947-9ce7-2506e0dbf037 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:48.980913 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a02103-07b2-4947-9ce7-2506e0dbf037 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:48.981405 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a02103-07b2-4947-9ce7-2506e0dbf037 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:48.982298 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a02103-07b2-4947-9ce7-2506e0dbf037 None admin] RBAC: Adding query filter params (interface=public, service_id=032ddc9dd0b44c66b74dc6406295b54e, region_id=RegionOne) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:48.982803 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a02103-07b2-4947-9ce7-2506e0dbf037 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:48.985422 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a02103-07b2-4947-9ce7-2506e0dbf037 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:48.997485 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 331/667] 10.222.0.26 () {56 vars in 1251 bytes} [Wed Nov 2 03:21:48 2022] GET /identity/v3/endpoints?service_id=032ddc9dd0b44c66b74dc6406295b54e&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:21:48.999100 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a02103-07b2-4947-9ce7-2506e0dbf037 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:49.056555 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:49.083729 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aee6566-830a-4df0-84f8-495f3e6821df None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:49.118236 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:49.128106 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841095a1-cfba-4f1a-aca8-a39e147d8c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:49.129053 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841095a1-cfba-4f1a-aca8-a39e147d8c3f None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:49.129346 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841095a1-cfba-4f1a-aca8-a39e147d8c3f None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:49.129618 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841095a1-cfba-4f1a-aca8-a39e147d8c3f None admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:49.130311 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841095a1-cfba-4f1a-aca8-a39e147d8c3f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:49.130615 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841095a1-cfba-4f1a-aca8-a39e147d8c3f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:49.131917 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a02103-07b2-4947-9ce7-2506e0dbf037 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:49.132156 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841095a1-cfba-4f1a-aca8-a39e147d8c3f None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:49.146184 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 337/668] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Nov 2 03:21:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:49.152539 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841095a1-cfba-4f1a-aca8-a39e147d8c3f None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:49.180999 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d547032-275c-4df3-b2a0-ef7e3a66d723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:49.181613 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d547032-275c-4df3-b2a0-ef7e3a66d723 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:49.181890 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d547032-275c-4df3-b2a0-ef7e3a66d723 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:49.182160 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d547032-275c-4df3-b2a0-ef7e3a66d723 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:49.182760 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d547032-275c-4df3-b2a0-ef7e3a66d723 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:49.184403 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d547032-275c-4df3-b2a0-ef7e3a66d723 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:49.190915 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 332/669] 10.222.0.26 () {56 vars in 1087 bytes} [Wed Nov 2 03:21:48 2022] GET /identity/v3/services => generated 1258 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 03:21:49.276434 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841095a1-cfba-4f1a-aca8-a39e147d8c3f None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:49.315979 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42283818-62a1-4cc9-a841-b02b7122bc31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:49.316659 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42283818-62a1-4cc9-a841-b02b7122bc31 None admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:49.316935 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42283818-62a1-4cc9-a841-b02b7122bc31 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:49.317207 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42283818-62a1-4cc9-a841-b02b7122bc31 None admin] PATH_INFO: `/v3/projects/d0b5e7017ac64ae6ab62c1a05aa54b72/users/da7ee607b66e43f3823d9d2faf373974/roles/c92260a73f5748618d4d4cbae46e5777` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:49.319966 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42283818-62a1-4cc9-a841-b02b7122bc31 None admin] RBAC: Authorizing `identity:create_grant(project_id=d0b5e7017ac64ae6ab62c1a05aa54b72, user_id=da7ee607b66e43f3823d9d2faf373974, role_id=c92260a73f5748618d4d4cbae46e5777)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:49.321485 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42283818-62a1-4cc9-a841-b02b7122bc31 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:49.338955 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 338/670] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Nov 2 03:21:49 2022] PUT /identity/v3/projects/d0b5e7017ac64ae6ab62c1a05aa54b72/users/da7ee607b66e43f3823d9d2faf373974/roles/c92260a73f5748618d4d4cbae46e5777 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 03:21:49.483174 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5062c-748b-4f35-b257-06d30f029e62 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:49.483587 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5062c-748b-4f35-b257-06d30f029e62 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:49.483724 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb5062c-748b-4f35-b257-06d30f029e62 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:49.485457 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 333/671] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:21:49.491806 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12783ab0-9868-4bf0-8901-b48bb9f3e10b None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:49.492471 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12783ab0-9868-4bf0-8901-b48bb9f3e10b None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:49.492982 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12783ab0-9868-4bf0-8901-b48bb9f3e10b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:49.513454 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-12783ab0-9868-4bf0-8901-b48bb9f3e10b None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:49.515003 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-12783ab0-9868-4bf0-8901-b48bb9f3e10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:49.566876 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:49.593742 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 339/672] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:49 2022] POST /identity/v3/auth/tokens => generated 1373 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 03:21:49.861460 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb5062c-748b-4f35-b257-06d30f029e62 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:49.967848 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb5062c-748b-4f35-b257-06d30f029e62 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:49.977296 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:50.003500 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7a318b-9dd2-456c-9e72-75f81f96b928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:50.004141 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7a318b-9dd2-456c-9e72-75f81f96b928 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:50.004422 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7a318b-9dd2-456c-9e72-75f81f96b928 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:50.004693 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7a318b-9dd2-456c-9e72-75f81f96b928 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:50.005352 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7a318b-9dd2-456c-9e72-75f81f96b928 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:50.006766 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7a318b-9dd2-456c-9e72-75f81f96b928 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:50.024056 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 334/673] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Nov 2 03:21:49 2022] POST /identity/v3/services => generated 251 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:21:50.908164 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a009e-6f56-4aa9-a562-6cf597eb9e69 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:50.909159 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a009e-6f56-4aa9-a562-6cf597eb9e69 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:50.909670 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a009e-6f56-4aa9-a562-6cf597eb9e69 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:50.911451 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 340/674] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:21:50.917665 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af93bb9-fbe1-42ce-8680-7462df9f974b None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:50.918113 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af93bb9-fbe1-42ce-8680-7462df9f974b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:50.918196 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af93bb9-fbe1-42ce-8680-7462df9f974b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:50.938680 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-5af93bb9-fbe1-42ce-8680-7462df9f974b None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:50.939910 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5af93bb9-fbe1-42ce-8680-7462df9f974b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:51.020313 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 335/675] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:50 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 03:21:51.291420 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85a009e-6f56-4aa9-a562-6cf597eb9e69 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:51.407910 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85a009e-6f56-4aa9-a562-6cf597eb9e69 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:51.430016 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7ee868-6a6c-4fab-b71d-3cab1022b4cb None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:51.430298 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7ee868-6a6c-4fab-b71d-3cab1022b4cb None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:51.430575 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7ee868-6a6c-4fab-b71d-3cab1022b4cb None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:51.435791 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 336/676] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:21:51.436957 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6721e3dc-653b-4ff6-8527-c71c39cedda8 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:51.437236 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6721e3dc-653b-4ff6-8527-c71c39cedda8 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:51.437509 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6721e3dc-653b-4ff6-8527-c71c39cedda8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:51.447011 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae055af7-55ad-48c6-b323-b9e817e98e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:51.447871 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae055af7-55ad-48c6-b323-b9e817e98e30 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:51.448339 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae055af7-55ad-48c6-b323-b9e817e98e30 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:51.448757 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae055af7-55ad-48c6-b323-b9e817e98e30 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:51.449487 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae055af7-55ad-48c6-b323-b9e817e98e30 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:51.450979 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae055af7-55ad-48c6-b323-b9e817e98e30 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:51.455630 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-6721e3dc-653b-4ff6-8527-c71c39cedda8 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:51.456879 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6721e3dc-653b-4ff6-8527-c71c39cedda8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:51.461283 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-ae055af7-55ad-48c6-b323-b9e817e98e30 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 03:21:51.461283 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:51.461283 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 03:21:51.461283 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 03:21:51.461283 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 03:21:51.461283 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 03:21:51.461283 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 03:21:51.461283 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 03:21:51.461283 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 03:21:51.461283 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:51.461283 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 03:21:51.461283 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:51.461283 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:51.461283 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:51.461283 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:51.461283 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:51.461283 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:51.462452 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:51.462452 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:51.462452 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:51.462452 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:51.462452 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:51.462452 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:51.462452 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 03:21:51.462452 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 03:21:51.462452 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 03:21:51.462452 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 03:21:51.462452 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:51.462452 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:51.462452 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:51.462452 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:51.462452 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:51.462452 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:51.463495 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:51.463495 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:51.463495 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:51.463495 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:51.463495 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:51.463495 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:51.463495 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:51.463495 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:51.463495 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:51.463495 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:51.463495 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 03:21:51.463495 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 03:21:51.463495 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 03:21:51.463495 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:51.465953 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 341/677] 10.222.0.26 () {56 vars in 1126 bytes} [Wed Nov 2 03:21:51 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:51.472345 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae055af7-55ad-48c6-b323-b9e817e98e30 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:51.526425 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:51.549493 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 337/678] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 03:21:51.596085 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae055af7-55ad-48c6-b323-b9e817e98e30 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:51.601038 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a58463-acb3-4cee-ace3-186504a7a603 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:51.601323 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a58463-acb3-4cee-ace3-186504a7a603 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:51.601594 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a58463-acb3-4cee-ace3-186504a7a603 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:51.603177 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 338/679] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:21:51.608234 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c1361-c737-4b77-b58d-1647824eecb7 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:51.608709 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c1361-c737-4b77-b58d-1647824eecb7 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:51.609200 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c1361-c737-4b77-b58d-1647824eecb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:51.630358 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-aa3c1361-c737-4b77-b58d-1647824eecb7 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:51.632192 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-aa3c1361-c737-4b77-b58d-1647824eecb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:51.640153 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e87eef-66dc-45bf-81d6-8396275e5717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:51.641468 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e87eef-66dc-45bf-81d6-8396275e5717 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:51.641940 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e87eef-66dc-45bf-81d6-8396275e5717 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:51.642399 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e87eef-66dc-45bf-81d6-8396275e5717 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:51.643650 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e87eef-66dc-45bf-81d6-8396275e5717 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:51.644260 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e87eef-66dc-45bf-81d6-8396275e5717 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:51.646456 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e87eef-66dc-45bf-81d6-8396275e5717 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:51.652345 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:51.653460 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 342/680] 10.222.0.26 () {56 vars in 1122 bytes} [Wed Nov 2 03:21:51 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:21:51.659425 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e87eef-66dc-45bf-81d6-8396275e5717 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:51.717742 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 339/681] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:51 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 03:21:51.780462 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e87eef-66dc-45bf-81d6-8396275e5717 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:51.788500 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:51.818607 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2bd92c-9936-4681-bc8a-0456a6f62160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:51.819985 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2bd92c-9936-4681-bc8a-0456a6f62160 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:51.820449 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2bd92c-9936-4681-bc8a-0456a6f62160 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:51.820890 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2bd92c-9936-4681-bc8a-0456a6f62160 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:51.821660 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2bd92c-9936-4681-bc8a-0456a6f62160 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:51.822106 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2bd92c-9936-4681-bc8a-0456a6f62160 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:51.823607 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2bd92c-9936-4681-bc8a-0456a6f62160 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:51.829269 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 343/682] 10.222.0.26 () {56 vars in 1122 bytes} [Wed Nov 2 03:21:51 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:51.837328 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3c1361-c737-4b77-b58d-1647824eecb7 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:51.853686 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2bd92c-9936-4681-bc8a-0456a6f62160 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:51.957378 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3c1361-c737-4b77-b58d-1647824eecb7 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:51.960554 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2bd92c-9936-4681-bc8a-0456a6f62160 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:51.996608 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4ee8cc-bd5c-4ea8-bd97-0bee83ff62d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:51.997048 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837f3c94-95a0-41ec-933a-a0923c6d1d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:51.997238 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4ee8cc-bd5c-4ea8-bd97-0bee83ff62d3 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:51.997518 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4ee8cc-bd5c-4ea8-bd97-0bee83ff62d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:51.997790 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4ee8cc-bd5c-4ea8-bd97-0bee83ff62d3 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:51.998367 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4ee8cc-bd5c-4ea8-bd97-0bee83ff62d3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:51.998572 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f3c94-95a0-41ec-933a-a0923c6d1d9d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:51.999017 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f3c94-95a0-41ec-933a-a0923c6d1d9d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:51.999470 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f3c94-95a0-41ec-933a-a0923c6d1d9d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:51.999723 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4ee8cc-bd5c-4ea8-bd97-0bee83ff62d3 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:52.000728 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837f3c94-95a0-41ec-933a-a0923c6d1d9d None admin] RBAC: Adding query filter params (interface=public, service_id=6a28dd0bc30d4fab838f2c55847008bb, region_id=RegionOne) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:52.001226 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837f3c94-95a0-41ec-933a-a0923c6d1d9d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:52.002742 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837f3c94-95a0-41ec-933a-a0923c6d1d9d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:52.008328 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 340/683] 10.222.0.26 () {56 vars in 1251 bytes} [Wed Nov 2 03:21:51 2022] GET /identity/v3/endpoints?service_id=6a28dd0bc30d4fab838f2c55847008bb&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:52.008702 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-cf4ee8cc-bd5c-4ea8-bd97-0bee83ff62d3 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 03:21:52.008702 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:52.008702 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 03:21:52.008702 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 03:21:52.008702 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 03:21:52.008702 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 03:21:52.008702 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 03:21:52.008702 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 03:21:52.008702 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 03:21:52.008702 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:52.008702 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 03:21:52.008702 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:52.008702 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:52.008702 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:52.008702 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:52.008702 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:52.009697 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:52.009697 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:52.009697 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:52.009697 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:52.009697 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:52.009697 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:52.009697 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:52.009697 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 03:21:52.009697 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 03:21:52.009697 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 03:21:52.009697 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 03:21:52.009697 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:52.009697 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:52.009697 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:52.009697 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:52.009697 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:52.009697 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:52.015702 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:52.015702 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:52.015702 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:52.015702 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:52.015702 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:52.015702 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:52.015702 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:52.015702 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:52.015702 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:52.015702 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:52.015702 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 03:21:52.015702 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 03:21:52.015702 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Nov 02 03:21:52.015702 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:52.015702 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 344/684] 10.222.0.26 () {56 vars in 1132 bytes} [Wed Nov 2 03:21:51 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 03:21:52.016564 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837f3c94-95a0-41ec-933a-a0923c6d1d9d None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:52.016637 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4ee8cc-bd5c-4ea8-bd97-0bee83ff62d3 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:52.036368 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:52.050913 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:52.122572 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4ee8cc-bd5c-4ea8-bd97-0bee83ff62d3 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:52.127407 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837f3c94-95a0-41ec-933a-a0923c6d1d9d None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:52.160682 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda45eb1-0388-4d8e-9f06-a06af573c1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:52.161316 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda45eb1-0388-4d8e-9f06-a06af573c1b7 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:52.161596 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda45eb1-0388-4d8e-9f06-a06af573c1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:52.161872 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda45eb1-0388-4d8e-9f06-a06af573c1b7 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:52.162556 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda45eb1-0388-4d8e-9f06-a06af573c1b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:52.163988 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda45eb1-0388-4d8e-9f06-a06af573c1b7 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:52.167475 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15294545-f1c7-4fff-a20b-7fa68f528ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:52.168140 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15294545-f1c7-4fff-a20b-7fa68f528ef5 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:52.168418 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15294545-f1c7-4fff-a20b-7fa68f528ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:52.168684 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15294545-f1c7-4fff-a20b-7fa68f528ef5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:52.179977 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 345/685] 10.222.0.26 () {56 vars in 1087 bytes} [Wed Nov 2 03:21:52 2022] GET /identity/v3/services => generated 1497 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 03:21:52.179977 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda45eb1-0388-4d8e-9f06-a06af573c1b7 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:52.191131 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15294545-f1c7-4fff-a20b-7fa68f528ef5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:21:52.196660 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-15294545-f1c7-4fff-a20b-7fa68f528ef5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 03:21:52.196660 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:52.196660 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:52.196660 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:52.196660 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:52.196660 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:52.196660 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:52.196660 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:52.196660 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:52.196660 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:52.196660 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:52.196660 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:52.196660 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:21:52.196660 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:21:52.196660 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:21:52.196660 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:21:52.197843 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:21:52.197843 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:21:52.197843 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:21:52.197843 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:21:52.197843 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:52.197843 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:52.197843 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:52.197843 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:52.197843 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:52.197843 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:52.197843 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:52.197843 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:52.197843 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:52.197843 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:52.197843 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:52.197843 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:52.198759 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:52.198759 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:52.198759 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:52.198759 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:52.198759 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:21:52.198759 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:21:52.198759 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:21:52.198759 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:21:52.198759 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:21:52.198759 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:21:52.198759 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 03:21:52.198759 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:52.202958 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 341/686] 10.222.0.26 () {56 vars in 1120 bytes} [Wed Nov 2 03:21:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:52.204030 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15294545-f1c7-4fff-a20b-7fa68f528ef5 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:52.296106 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:52.298334 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:52.302465 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda45eb1-0388-4d8e-9f06-a06af573c1b7 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:52.324077 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15294545-f1c7-4fff-a20b-7fa68f528ef5 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:52.342490 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e066bc-062d-40ac-b5fa-a140b6688e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:52.343109 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e066bc-062d-40ac-b5fa-a140b6688e1c None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:52.343387 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e066bc-062d-40ac-b5fa-a140b6688e1c None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:52.343655 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e066bc-062d-40ac-b5fa-a140b6688e1c None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:52.344331 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e066bc-062d-40ac-b5fa-a140b6688e1c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:52.344636 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e066bc-062d-40ac-b5fa-a140b6688e1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:52.346042 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e066bc-062d-40ac-b5fa-a140b6688e1c None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:52.356925 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 346/687] 10.222.0.26 () {56 vars in 1126 bytes} [Wed Nov 2 03:21:52 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:52.356925 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a52068-5337-4992-a113-aec1607bdc1f None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:52.356925 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a52068-5337-4992-a113-aec1607bdc1f None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:52.356925 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a52068-5337-4992-a113-aec1607bdc1f None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:52.356925 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 347/688] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:21:52.360257 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a52068-5337-4992-a113-aec1607bdc1f None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:52.369817 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad01185b-564d-411e-ae0a-b4c9e9891433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:52.371147 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad01185b-564d-411e-ae0a-b4c9e9891433 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:52.371147 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad01185b-564d-411e-ae0a-b4c9e9891433 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:52.371147 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad01185b-564d-411e-ae0a-b4c9e9891433 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:52.371989 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad01185b-564d-411e-ae0a-b4c9e9891433 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:52.371989 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad01185b-564d-411e-ae0a-b4c9e9891433 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:52.373963 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad01185b-564d-411e-ae0a-b4c9e9891433 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:52.383685 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 342/689] 10.222.0.26 () {56 vars in 1115 bytes} [Wed Nov 2 03:21:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:52.385382 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e8670-9995-414c-978a-a1e16f2fd93d None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:52.385382 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e8670-9995-414c-978a-a1e16f2fd93d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:52.385382 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714e8670-9995-414c-978a-a1e16f2fd93d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:52.405337 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-714e8670-9995-414c-978a-a1e16f2fd93d None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:52.406520 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-714e8670-9995-414c-978a-a1e16f2fd93d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:52.469537 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a52068-5337-4992-a113-aec1607bdc1f None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:52.475706 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 343/690] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:52 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 03:21:52.476597 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714e8670-9995-414c-978a-a1e16f2fd93d None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:52.520207 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad76c1f5-7148-4c94-8552-fe3b61be7cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:52.521127 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76c1f5-7148-4c94-8552-fe3b61be7cc0 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:52.521587 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76c1f5-7148-4c94-8552-fe3b61be7cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:52.522682 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76c1f5-7148-4c94-8552-fe3b61be7cc0 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:52.523522 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad76c1f5-7148-4c94-8552-fe3b61be7cc0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:52.524029 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad76c1f5-7148-4c94-8552-fe3b61be7cc0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:52.526488 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad76c1f5-7148-4c94-8552-fe3b61be7cc0 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:52.547204 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 348/691] 10.222.0.26 () {56 vars in 1126 bytes} [Wed Nov 2 03:21:52 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:52.549206 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad76c1f5-7148-4c94-8552-fe3b61be7cc0 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:52.556386 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:52.587256 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:52.593876 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714e8670-9995-414c-978a-a1e16f2fd93d None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:52.637734 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8768e75-751a-460f-9f9e-37f38e36b443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:52.638366 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8768e75-751a-460f-9f9e-37f38e36b443 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:52.638644 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8768e75-751a-460f-9f9e-37f38e36b443 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:52.639183 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8768e75-751a-460f-9f9e-37f38e36b443 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:52.653417 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8768e75-751a-460f-9f9e-37f38e36b443 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:52.655565 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8768e75-751a-460f-9f9e-37f38e36b443 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:52.675990 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-a8768e75-751a-460f-9f9e-37f38e36b443 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:52.675990 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:52.675990 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:52.675990 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:52.675990 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:52.675990 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:52.675990 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:52.675990 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:52.675990 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:52.675990 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:52.675990 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:52.675990 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:52.675990 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 03:21:52.675990 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 03:21:52.675990 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 03:21:52.675990 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 03:21:52.677001 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:52.677001 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:52.677001 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:52.677001 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:52.677001 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:52.677001 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:52.677001 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:52.677001 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:52.677001 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:52.677001 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:52.677001 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:52.677001 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:52.677001 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:52.677001 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:52.677001 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:52.677001 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:52.677001 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 03:21:52.678583 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 03:21:52.678583 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:52.678583 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:52.680826 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad76c1f5-7148-4c94-8552-fe3b61be7cc0 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:52.682379 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 344/692] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:21:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:52.683574 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8768e75-751a-460f-9f9e-37f38e36b443 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:52.719058 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2777df33-d22c-4d47-b49d-e2859666b322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:52.720172 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2777df33-d22c-4d47-b49d-e2859666b322 None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:52.720687 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2777df33-d22c-4d47-b49d-e2859666b322 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:52.721165 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2777df33-d22c-4d47-b49d-e2859666b322 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:52.722083 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2777df33-d22c-4d47-b49d-e2859666b322 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:52.723713 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2777df33-d22c-4d47-b49d-e2859666b322 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:52.755975 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 349/693] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:21:52 2022] POST /identity/v3/endpoints => generated 347 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:21:52.763245 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2777df33-d22c-4d47-b49d-e2859666b322 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:52.810707 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8768e75-751a-460f-9f9e-37f38e36b443 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:52.838151 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:52.858253 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cec69e-aa5e-419b-acb2-bc9de6704b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:52.858969 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cec69e-aa5e-419b-acb2-bc9de6704b33 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:52.859354 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cec69e-aa5e-419b-acb2-bc9de6704b33 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:52.859687 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cec69e-aa5e-419b-acb2-bc9de6704b33 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:52.860552 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cec69e-aa5e-419b-acb2-bc9de6704b33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:52.860917 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cec69e-aa5e-419b-acb2-bc9de6704b33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:52.862609 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cec69e-aa5e-419b-acb2-bc9de6704b33 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:52.872728 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:52.877898 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 345/694] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Nov 2 03:21:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:52.885638 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cec69e-aa5e-419b-acb2-bc9de6704b33 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:52.902959 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2777df33-d22c-4d47-b49d-e2859666b322 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:52.948142 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd6008d-0be3-4e47-b421-989dcf9c617e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:52.948900 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd6008d-0be3-4e47-b421-989dcf9c617e None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:52.949185 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd6008d-0be3-4e47-b421-989dcf9c617e None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:52.949456 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd6008d-0be3-4e47-b421-989dcf9c617e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:52.950091 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd6008d-0be3-4e47-b421-989dcf9c617e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:52.951513 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd6008d-0be3-4e47-b421-989dcf9c617e None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:52.961369 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-fcd6008d-0be3-4e47-b421-989dcf9c617e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 03:21:52.961369 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:52.961369 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 03:21:52.961369 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 03:21:52.961369 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 03:21:52.961369 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 03:21:52.961369 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 03:21:52.961369 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 03:21:52.961369 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 03:21:52.961369 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:52.961369 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 03:21:52.961369 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:52.961369 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:52.961369 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:52.961369 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:52.961369 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:52.961369 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:52.962796 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:52.962796 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:52.962796 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:52.962796 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:52.962796 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:52.962796 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:52.962796 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 03:21:52.962796 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 03:21:52.962796 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 03:21:52.962796 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 03:21:52.962796 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:52.962796 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:52.962796 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:52.962796 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:52.962796 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:52.962796 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:52.963625 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:52.963625 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:52.963625 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:52.963625 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:52.963625 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:52.963625 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:52.963625 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:52.963625 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:52.963625 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:52.963625 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:52.963625 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 03:21:52.963625 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 03:21:52.963625 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 03:21:52.963625 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:52.963625 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 350/695] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Nov 2 03:21:52 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 03:21:52.969293 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd6008d-0be3-4e47-b421-989dcf9c617e None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:53.037065 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cec69e-aa5e-419b-acb2-bc9de6704b33 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:53.077422 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d091bdc3-cb43-4663-9e7b-a29ad1d84b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:53.078477 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d091bdc3-cb43-4663-9e7b-a29ad1d84b62 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:53.079058 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d091bdc3-cb43-4663-9e7b-a29ad1d84b62 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:53.079530 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d091bdc3-cb43-4663-9e7b-a29ad1d84b62 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:53.082959 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd6008d-0be3-4e47-b421-989dcf9c617e None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:53.087777 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d091bdc3-cb43-4663-9e7b-a29ad1d84b62 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:53.089516 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d091bdc3-cb43-4663-9e7b-a29ad1d84b62 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:53.103942 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-d091bdc3-cb43-4663-9e7b-a29ad1d84b62 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 03:21:53.103942 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:53.103942 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:53.103942 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:53.103942 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:53.103942 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:53.103942 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:53.103942 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:53.103942 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:53.103942 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:53.103942 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:53.103942 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:53.103942 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 03:21:53.103942 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 03:21:53.103942 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 03:21:53.103942 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 03:21:53.105008 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:53.105008 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:53.105008 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 03:21:53.105008 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 03:21:53.105008 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 03:21:53.105008 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 03:21:53.105008 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:53.105008 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:53.105008 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:53.105008 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:53.105008 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:53.105008 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:53.105008 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:53.105008 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:53.105008 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:53.105008 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:53.105008 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:53.105008 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:53.105962 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:53.105962 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:53.105962 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 03:21:53.105962 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 03:21:53.105962 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 03:21:53.105962 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 03:21:53.105962 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 03:21:53.105962 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 03:21:53.105962 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Nov 02 03:21:53.105962 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:53.113345 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 346/696] 10.222.0.26 () {56 vars in 1117 bytes} [Wed Nov 2 03:21:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:53.115081 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d091bdc3-cb43-4663-9e7b-a29ad1d84b62 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:53.123019 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43eddac-b9c0-4444-aeda-edef81502eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:53.123652 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43eddac-b9c0-4444-aeda-edef81502eb9 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:53.123926 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43eddac-b9c0-4444-aeda-edef81502eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:53.124225 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43eddac-b9c0-4444-aeda-edef81502eb9 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:53.124851 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43eddac-b9c0-4444-aeda-edef81502eb9 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:53.125150 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43eddac-b9c0-4444-aeda-edef81502eb9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:53.126505 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43eddac-b9c0-4444-aeda-edef81502eb9 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:53.132680 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 351/697] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Nov 2 03:21:52 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:21:53.138128 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43eddac-b9c0-4444-aeda-edef81502eb9 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:53.150891 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:53.163604 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:53.235217 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d091bdc3-cb43-4663-9e7b-a29ad1d84b62 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:53.268499 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43eddac-b9c0-4444-aeda-edef81502eb9 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:53.274965 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9898cf0c-ab4f-4c53-a782-c85968c71b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:53.275823 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898cf0c-ab4f-4c53-a782-c85968c71b19 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:53.276323 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898cf0c-ab4f-4c53-a782-c85968c71b19 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:53.276832 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898cf0c-ab4f-4c53-a782-c85968c71b19 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:53.277775 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9898cf0c-ab4f-4c53-a782-c85968c71b19 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:53.278282 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9898cf0c-ab4f-4c53-a782-c85968c71b19 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:53.279920 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9898cf0c-ab4f-4c53-a782-c85968c71b19 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:53.306001 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 347/698] 10.222.0.26 () {56 vars in 1149 bytes} [Wed Nov 2 03:21:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:21:53.307414 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9898cf0c-ab4f-4c53-a782-c85968c71b19 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:53.309210 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c78191-6696-431e-a4d9-4235cacca15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:53.310151 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c78191-6696-431e-a4d9-4235cacca15b None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:53.310427 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c78191-6696-431e-a4d9-4235cacca15b None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:53.310704 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c78191-6696-431e-a4d9-4235cacca15b None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:53.311458 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c78191-6696-431e-a4d9-4235cacca15b None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:53.311649 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c78191-6696-431e-a4d9-4235cacca15b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:53.313542 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c78191-6696-431e-a4d9-4235cacca15b None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:53.324185 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 352/699] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Nov 2 03:21:53 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:21:53.325086 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c78191-6696-431e-a4d9-4235cacca15b None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:53.403496 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:53.420917 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:53.429187 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9898cf0c-ab4f-4c53-a782-c85968c71b19 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:53.432898 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c78191-6696-431e-a4d9-4235cacca15b None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:53.467610 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6aa5af-e19b-4ce7-952c-2414a23c9268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:53.468811 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6aa5af-e19b-4ce7-952c-2414a23c9268 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:53.469339 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6aa5af-e19b-4ce7-952c-2414a23c9268 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:53.469876 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6aa5af-e19b-4ce7-952c-2414a23c9268 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:53.471838 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db84ab-7d1d-4932-89f7-6c234b33a75f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:53.472669 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db84ab-7d1d-4932-89f7-6c234b33a75f None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:53.473011 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db84ab-7d1d-4932-89f7-6c234b33a75f None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:53.473411 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db84ab-7d1d-4932-89f7-6c234b33a75f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:53.474338 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19db84ab-7d1d-4932-89f7-6c234b33a75f None admin] RBAC: Adding query filter params (interface=public, service_id=f8e0b4fedafd460da44e5f6f1b423548, region_id=RegionOne) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:53.474710 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19db84ab-7d1d-4932-89f7-6c234b33a75f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:53.476499 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19db84ab-7d1d-4932-89f7-6c234b33a75f None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:53.486948 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 353/700] 10.222.0.26 () {56 vars in 1251 bytes} [Wed Nov 2 03:21:53 2022] GET /identity/v3/endpoints?service_id=f8e0b4fedafd460da44e5f6f1b423548&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:53.488307 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db84ab-7d1d-4932-89f7-6c234b33a75f None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:53.553904 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6aa5af-e19b-4ce7-952c-2414a23c9268 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:53.555670 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6aa5af-e19b-4ce7-952c-2414a23c9268 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:53.612897 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db84ab-7d1d-4932-89f7-6c234b33a75f None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:53.652683 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6407786-cb15-4a44-9462-efad6a4a6cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:53.653394 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6407786-cb15-4a44-9462-efad6a4a6cc4 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:53.653734 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6407786-cb15-4a44-9462-efad6a4a6cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:53.654075 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6407786-cb15-4a44-9462-efad6a4a6cc4 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:53.654736 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6407786-cb15-4a44-9462-efad6a4a6cc4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:53.656323 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6407786-cb15-4a44-9462-efad6a4a6cc4 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:53.660147 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:53.667677 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 348/702] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Nov 2 03:21:53 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 03:21:53.672289 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 354/702] 10.222.0.26 () {56 vars in 1087 bytes} [Wed Nov 2 03:21:53 2022] GET /identity/v3/services => generated 1497 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:21:53.673556 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6aa5af-e19b-4ce7-952c-2414a23c9268 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:53.788228 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6aa5af-e19b-4ce7-952c-2414a23c9268 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:53.826785 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977667fe-6bac-4c3e-964a-b60c84493ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:53.827861 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977667fe-6bac-4c3e-964a-b60c84493ccd None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:53.828389 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977667fe-6bac-4c3e-964a-b60c84493ccd None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:53.828866 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977667fe-6bac-4c3e-964a-b60c84493ccd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:53.840517 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977667fe-6bac-4c3e-964a-b60c84493ccd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:53.842377 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977667fe-6bac-4c3e-964a-b60c84493ccd None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:53.858398 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-977667fe-6bac-4c3e-964a-b60c84493ccd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:53.858398 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:53.858398 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:53.858398 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:53.858398 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:53.858398 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:53.858398 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:53.858398 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:53.858398 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:53.858398 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:53.858398 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:53.858398 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:53.858398 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 03:21:53.858398 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 03:21:53.858398 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 03:21:53.858398 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 03:21:53.859363 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:53.859363 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:53.859363 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:53.859363 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:53.859363 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:53.859363 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:53.859363 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:53.859363 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:53.859363 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:53.859363 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:53.859363 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:53.859363 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:53.859363 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:53.859363 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:53.859363 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:53.859363 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:53.859363 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 03:21:53.860349 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 03:21:53.860349 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:21:53.860349 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:53.860349 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 349/703] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:21:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:53.865489 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6407786-cb15-4a44-9462-efad6a4a6cc4 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:53.868440 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:53.976529 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6407786-cb15-4a44-9462-efad6a4a6cc4 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:54.013181 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f69d42c-f6be-4e60-9a9a-800e3241392b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:54.013798 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f69d42c-f6be-4e60-9a9a-800e3241392b None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:54.014080 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f69d42c-f6be-4e60-9a9a-800e3241392b None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:54.014353 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f69d42c-f6be-4e60-9a9a-800e3241392b None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:54.015002 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f69d42c-f6be-4e60-9a9a-800e3241392b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:54.015307 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f69d42c-f6be-4e60-9a9a-800e3241392b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:54.016709 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f69d42c-f6be-4e60-9a9a-800e3241392b None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:54.029810 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 355/704] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Nov 2 03:21:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:21:54.035041 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977667fe-6bac-4c3e-964a-b60c84493ccd None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:54.086704 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:54.139879 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977667fe-6bac-4c3e-964a-b60c84493ccd None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:54.174552 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf3f62a-75f5-42e8-a42f-eb8fc4e2c014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:54.175188 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf3f62a-75f5-42e8-a42f-eb8fc4e2c014 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:54.175469 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf3f62a-75f5-42e8-a42f-eb8fc4e2c014 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:54.175743 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf3f62a-75f5-42e8-a42f-eb8fc4e2c014 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:54.183267 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf3f62a-75f5-42e8-a42f-eb8fc4e2c014 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:54.184741 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf3f62a-75f5-42e8-a42f-eb8fc4e2c014 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:54.196150 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-4cf3f62a-75f5-42e8-a42f-eb8fc4e2c014 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 03:21:54.196150 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:54.196150 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:54.196150 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:54.196150 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:54.196150 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:54.196150 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:54.196150 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:54.196150 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:54.196150 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:54.196150 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:54.196150 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:54.196150 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 03:21:54.196150 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 03:21:54.196150 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 03:21:54.197016 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 03:21:54.197016 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:54.197016 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:54.197016 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:54.197016 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:54.197016 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:54.197016 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:54.197016 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:54.197016 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:54.197016 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:54.197016 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:54.197016 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:54.197016 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:54.197016 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:54.197016 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:54.197016 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:54.197016 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:54.197939 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 03:21:54.197939 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 03:21:54.197939 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 03:21:54.197939 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 03:21:54.197939 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 03:21:54.197939 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 03:21:54.197939 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 03:21:54.197939 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:54.197939 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 350/705] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Nov 2 03:21:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:54.203366 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f69d42c-f6be-4e60-9a9a-800e3241392b None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:54.268715 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:54.315212 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f69d42c-f6be-4e60-9a9a-800e3241392b None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:54.350972 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e4a196-4e92-499f-927a-cbe6ed4934a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:54.351590 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e4a196-4e92-499f-927a-cbe6ed4934a9 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:54.351872 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e4a196-4e92-499f-927a-cbe6ed4934a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:54.352174 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e4a196-4e92-499f-927a-cbe6ed4934a9 None admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:54.352925 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e4a196-4e92-499f-927a-cbe6ed4934a9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:54.353236 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e4a196-4e92-499f-927a-cbe6ed4934a9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:54.354760 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e4a196-4e92-499f-927a-cbe6ed4934a9 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:54.368583 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 356/706] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Nov 2 03:21:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:54.374317 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf3f62a-75f5-42e8-a42f-eb8fc4e2c014 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:54.397737 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3273c95f-494f-4d3b-bbf6-5798e37be18a None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:54.398022 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3273c95f-494f-4d3b-bbf6-5798e37be18a None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:54.398293 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3273c95f-494f-4d3b-bbf6-5798e37be18a None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:54.400935 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 357/707] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 03:21:54.404869 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb34116-633d-4893-a532-767eda96d97a None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:54.405150 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb34116-633d-4893-a532-767eda96d97a None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:54.405424 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb34116-633d-4893-a532-767eda96d97a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:54.423497 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-edb34116-633d-4893-a532-767eda96d97a None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:54.424707 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-edb34116-633d-4893-a532-767eda96d97a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:54.481509 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf3f62a-75f5-42e8-a42f-eb8fc4e2c014 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:54.492003 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:54.516093 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 358/708] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 03:21:54.524653 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8151902-03ba-40f4-a0c7-4cd481f68a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:54.524905 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8151902-03ba-40f4-a0c7-4cd481f68a11 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:54.525139 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8151902-03ba-40f4-a0c7-4cd481f68a11 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:54.525410 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8151902-03ba-40f4-a0c7-4cd481f68a11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:54.526198 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8151902-03ba-40f4-a0c7-4cd481f68a11 None admin] RBAC: Adding query filter params (role.id=c92260a73f5748618d4d4cbae46e5777, scope.project.id=d0b5e7017ac64ae6ab62c1a05aa54b72, user.id=da7ee607b66e43f3823d9d2faf373974) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:54.526501 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8151902-03ba-40f4-a0c7-4cd481f68a11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:54.527991 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8151902-03ba-40f4-a0c7-4cd481f68a11 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:54.535139 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 351/709] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Nov 2 03:21:54 2022] GET /identity/v3/role_assignments?user.id=da7ee607b66e43f3823d9d2faf373974&scope.project.id=d0b5e7017ac64ae6ab62c1a05aa54b72&role.id=c92260a73f5748618d4d4cbae46e5777 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:54.786137 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb34116-633d-4893-a532-767eda96d97a None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:54.840401 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:54.888551 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb34116-633d-4893-a532-767eda96d97a None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:54.927467 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ea3323-5fd8-4a7b-8345-46307f4ea798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:54.928708 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea3323-5fd8-4a7b-8345-46307f4ea798 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:54.928991 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea3323-5fd8-4a7b-8345-46307f4ea798 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:54.929269 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ea3323-5fd8-4a7b-8345-46307f4ea798 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:54.929857 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ea3323-5fd8-4a7b-8345-46307f4ea798 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:54.931217 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ea3323-5fd8-4a7b-8345-46307f4ea798 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:54.940285 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-06ea3323-5fd8-4a7b-8345-46307f4ea798 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 03:21:54.940285 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:54.940285 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 03:21:54.940285 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 03:21:54.940285 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 03:21:54.940285 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 03:21:54.940285 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 03:21:54.940285 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 03:21:54.940285 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 03:21:54.940285 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:54.940285 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 03:21:54.940285 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:54.940285 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:54.940285 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:54.940285 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:54.940285 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:54.940285 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:54.941290 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:54.941290 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:54.941290 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:54.941290 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:54.941290 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:54.941290 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:54.941290 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 03:21:54.941290 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 03:21:54.941290 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 03:21:54.941290 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 03:21:54.941290 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:54.941290 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:54.941290 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:54.941290 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:54.941290 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:54.941290 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:54.942150 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:54.942150 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:54.942150 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:54.942150 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:54.942150 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:54.942150 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:54.942150 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:54.942150 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:54.942150 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:54.942150 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:54.942150 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 03:21:54.942150 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 03:21:54.942150 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 03:21:54.942150 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:54.942150 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 359/710] 10.222.0.26 () {56 vars in 1126 bytes} [Wed Nov 2 03:21:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:54.947527 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8151902-03ba-40f4-a0c7-4cd481f68a11 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:55.058694 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8151902-03ba-40f4-a0c7-4cd481f68a11 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:55.093999 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104d416a-b74e-4ce5-a44f-e33e6b1d52de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:55.094771 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d416a-b74e-4ce5-a44f-e33e6b1d52de None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:55.094933 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d416a-b74e-4ce5-a44f-e33e6b1d52de None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:55.095081 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35208adc-338e-4e66-a531-df0d0befcb27 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:55.095497 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104d416a-b74e-4ce5-a44f-e33e6b1d52de None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:55.095722 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35208adc-338e-4e66-a531-df0d0befcb27 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:55.095722 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35208adc-338e-4e66-a531-df0d0befcb27 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:55.096035 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104d416a-b74e-4ce5-a44f-e33e6b1d52de None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:55.096035 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104d416a-b74e-4ce5-a44f-e33e6b1d52de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:55.097050 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 360/711] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 03:21:55.097461 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104d416a-b74e-4ce5-a44f-e33e6b1d52de None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:55.102060 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a7ea21-3480-48f1-83d0-222a005c3cf0 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:55.102333 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a7ea21-3480-48f1-83d0-222a005c3cf0 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:55.102604 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a7ea21-3480-48f1-83d0-222a005c3cf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:55.107196 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 352/712] 10.222.0.26 () {56 vars in 1122 bytes} [Wed Nov 2 03:21:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:55.108384 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104d416a-b74e-4ce5-a44f-e33e6b1d52de None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:55.122197 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-f7a7ea21-3480-48f1-83d0-222a005c3cf0 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:55.123357 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f7a7ea21-3480-48f1-83d0-222a005c3cf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:55.145277 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:55.220008 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104d416a-b74e-4ce5-a44f-e33e6b1d52de None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:55.220199 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 361/713] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 03:21:55.254078 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fefcbd5-0725-4946-8067-9237af4153f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:55.254681 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fefcbd5-0725-4946-8067-9237af4153f7 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:55.254953 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fefcbd5-0725-4946-8067-9237af4153f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:55.255219 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fefcbd5-0725-4946-8067-9237af4153f7 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:55.255838 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fefcbd5-0725-4946-8067-9237af4153f7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:55.256218 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fefcbd5-0725-4946-8067-9237af4153f7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:55.257641 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fefcbd5-0725-4946-8067-9237af4153f7 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:55.267192 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 353/714] 10.222.0.26 () {56 vars in 1122 bytes} [Wed Nov 2 03:21:55 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:21:55.268569 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a7ea21-3480-48f1-83d0-222a005c3cf0 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:55.315852 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:55.377135 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a7ea21-3480-48f1-83d0-222a005c3cf0 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:55.412898 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb1031-64bc-4f55-b46b-b164c02e84ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:55.413866 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb1031-64bc-4f55-b46b-b164c02e84ac None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:55.414393 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb1031-64bc-4f55-b46b-b164c02e84ac None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:55.414863 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79eb1031-64bc-4f55-b46b-b164c02e84ac None admin] PATH_INFO: `/v3/endpoints` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:55.415815 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eb1031-64bc-4f55-b46b-b164c02e84ac None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:55.417380 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79eb1031-64bc-4f55-b46b-b164c02e84ac None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:55.440533 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 362/715] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:21:55 2022] POST /identity/v3/endpoints => generated 349 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:21:55.515897 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fefcbd5-0725-4946-8067-9237af4153f7 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:55.629629 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:55.633432 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fefcbd5-0725-4946-8067-9237af4153f7 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:55.674054 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f0b851-9b3c-4924-8220-f2c60bfa5838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:55.674689 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f0b851-9b3c-4924-8220-f2c60bfa5838 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:55.674966 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f0b851-9b3c-4924-8220-f2c60bfa5838 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:55.675235 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f0b851-9b3c-4924-8220-f2c60bfa5838 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:55.675803 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f0b851-9b3c-4924-8220-f2c60bfa5838 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:55.677184 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f0b851-9b3c-4924-8220-f2c60bfa5838 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:55.686568 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-93f0b851-9b3c-4924-8220-f2c60bfa5838 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 03:21:55.686568 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:55.686568 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 03:21:55.686568 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 03:21:55.686568 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 03:21:55.686568 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 03:21:55.686568 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 03:21:55.686568 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 03:21:55.686568 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 03:21:55.686568 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:55.686568 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 03:21:55.686568 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:55.686568 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:55.686568 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:55.686568 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:55.686568 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:55.686568 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:55.687558 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:55.687558 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:55.687558 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:55.687558 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:55.687558 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:55.687558 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:55.687558 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 03:21:55.687558 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 03:21:55.687558 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 03:21:55.687558 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 03:21:55.687558 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:55.687558 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:55.687558 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:55.687558 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:55.687558 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:55.687558 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:55.688443 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:55.688443 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:55.688443 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:55.688443 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:55.688443 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:55.688443 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:55.688443 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:55.688443 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:55.688443 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:55.688443 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:55.688443 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 03:21:55.688443 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 03:21:55.688443 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 03:21:55.688443 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:55.688443 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 354/716] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Nov 2 03:21:55 2022] GET /identity/v3/services/compute => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:55.693940 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb1031-64bc-4f55-b46b-b164c02e84ac None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:55.797856 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79eb1031-64bc-4f55-b46b-b164c02e84ac None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:55.799498 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:55.835158 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4a3538-d21c-403f-9aba-cf7a838c49c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:55.835772 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a3538-d21c-403f-9aba-cf7a838c49c3 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:55.836123 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a3538-d21c-403f-9aba-cf7a838c49c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:55.836392 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4a3538-d21c-403f-9aba-cf7a838c49c3 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:55.837016 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4a3538-d21c-403f-9aba-cf7a838c49c3 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:55.837361 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4a3538-d21c-403f-9aba-cf7a838c49c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:55.838721 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4a3538-d21c-403f-9aba-cf7a838c49c3 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:55.843846 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 363/717] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Nov 2 03:21:55 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:55.849559 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f0b851-9b3c-4924-8220-f2c60bfa5838 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:55.958682 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f0b851-9b3c-4924-8220-f2c60bfa5838 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:55.994740 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f82dec2-dfab-4f20-9d76-737af58030a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:55.995350 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f82dec2-dfab-4f20-9d76-737af58030a0 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:55.995626 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f82dec2-dfab-4f20-9d76-737af58030a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:55.995895 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f82dec2-dfab-4f20-9d76-737af58030a0 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:55.996561 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f82dec2-dfab-4f20-9d76-737af58030a0 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:55.996859 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f82dec2-dfab-4f20-9d76-737af58030a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:55.998209 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f82dec2-dfab-4f20-9d76-737af58030a0 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:56.006623 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 355/718] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Nov 2 03:21:55 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:56.006895 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe669a3-d652-4d7a-9f7e-c534cedeaae8 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:56.006895 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe669a3-d652-4d7a-9f7e-c534cedeaae8 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:56.006895 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe669a3-d652-4d7a-9f7e-c534cedeaae8 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:56.007599 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 364/719] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:21:56.013549 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7b164-5c1c-496c-a068-19f6909d9c27 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:56.014171 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7b164-5c1c-496c-a068-19f6909d9c27 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:56.014651 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c7b164-5c1c-496c-a068-19f6909d9c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:56.039102 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-88c7b164-5c1c-496c-a068-19f6909d9c27 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:56.040348 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-88c7b164-5c1c-496c-a068-19f6909d9c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:56.072361 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:56.116303 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 356/720] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 03:21:56.388348 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe669a3-d652-4d7a-9f7e-c534cedeaae8 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:56.493481 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe669a3-d652-4d7a-9f7e-c534cedeaae8 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:56.542168 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a62ff62-16b6-46da-899e-f53b4748c7aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:56.543054 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a62ff62-16b6-46da-899e-f53b4748c7aa None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:56.543486 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a62ff62-16b6-46da-899e-f53b4748c7aa None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:56.543909 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a62ff62-16b6-46da-899e-f53b4748c7aa None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:56.544688 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a62ff62-16b6-46da-899e-f53b4748c7aa None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:56.546197 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a62ff62-16b6-46da-899e-f53b4748c7aa None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:56.556258 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-8a62ff62-16b6-46da-899e-f53b4748c7aa None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 03:21:56.556258 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:56.556258 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 03:21:56.556258 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 03:21:56.556258 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 03:21:56.556258 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 03:21:56.556258 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 03:21:56.556258 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 03:21:56.556258 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 03:21:56.556258 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:56.556258 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 03:21:56.556258 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:56.556258 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:56.556258 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:56.556258 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:56.556258 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:56.556258 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:56.557455 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:56.557455 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:56.557455 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:56.557455 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:56.557455 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:56.557455 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:56.557455 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 03:21:56.557455 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 03:21:56.557455 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 03:21:56.557455 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 03:21:56.557455 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:56.557455 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:56.557455 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:56.557455 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:56.557455 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:56.557455 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:56.558490 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:56.558490 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:56.558490 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:56.558490 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:56.558490 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:56.558490 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:56.558490 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:56.558490 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:56.558490 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:56.558490 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:56.558490 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 03:21:56.558490 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 03:21:56.558490 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Nov 02 03:21:56.558490 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:56.560875 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 365/721] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Nov 2 03:21:56 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:56.567085 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c7b164-5c1c-496c-a068-19f6909d9c27 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:56.675380 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c7b164-5c1c-496c-a068-19f6909d9c27 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:56.710381 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87594250-3744-4d25-b44e-0603fb7d45e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:56.710997 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87594250-3744-4d25-b44e-0603fb7d45e8 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:56.711273 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87594250-3744-4d25-b44e-0603fb7d45e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:56.711540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87594250-3744-4d25-b44e-0603fb7d45e8 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:56.712194 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87594250-3744-4d25-b44e-0603fb7d45e8 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:56.712497 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87594250-3744-4d25-b44e-0603fb7d45e8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:56.713845 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87594250-3744-4d25-b44e-0603fb7d45e8 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:56.719195 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 357/722] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Nov 2 03:21:56 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:56.724718 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a62ff62-16b6-46da-899e-f53b4748c7aa None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:56.757648 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9480cc8f-4764-4192-b839-c8b38e191195 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:56.757961 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9480cc8f-4764-4192-b839-c8b38e191195 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:56.758207 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9480cc8f-4764-4192-b839-c8b38e191195 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:56.759677 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 358/723] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 03:21:56.763742 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:56.764978 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed66cd3a-85d0-4c98-82e9-823d83aa728e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:56.765261 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed66cd3a-85d0-4c98-82e9-823d83aa728e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:56.765530 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed66cd3a-85d0-4c98-82e9-823d83aa728e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:56.784461 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-ed66cd3a-85d0-4c98-82e9-823d83aa728e None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:56.785647 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ed66cd3a-85d0-4c98-82e9-823d83aa728e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:56.800356 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:56.855906 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a62ff62-16b6-46da-899e-f53b4748c7aa None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:56.859682 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 359/724] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 03:21:56.891421 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b5db6-bd5d-428a-81dd-3646fc4cb53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:56.892322 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b5db6-bd5d-428a-81dd-3646fc4cb53a None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:56.892742 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b5db6-bd5d-428a-81dd-3646fc4cb53a None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:56.893155 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44b5db6-bd5d-428a-81dd-3646fc4cb53a None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:56.893929 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44b5db6-bd5d-428a-81dd-3646fc4cb53a None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:56.894388 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44b5db6-bd5d-428a-81dd-3646fc4cb53a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:56.895875 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44b5db6-bd5d-428a-81dd-3646fc4cb53a None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:56.901248 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 366/725] 10.222.0.26 () {56 vars in 1114 bytes} [Wed Nov 2 03:21:56 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:56.906587 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed66cd3a-85d0-4c98-82e9-823d83aa728e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:57.016656 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed66cd3a-85d0-4c98-82e9-823d83aa728e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:57.051788 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8022db6-c07f-4b5a-8fe8-781fe8527536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:57.052428 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8022db6-c07f-4b5a-8fe8-781fe8527536 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:57.052705 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8022db6-c07f-4b5a-8fe8-781fe8527536 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:57.053030 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8022db6-c07f-4b5a-8fe8-781fe8527536 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:57.053692 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8022db6-c07f-4b5a-8fe8-781fe8527536 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:57.055058 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8022db6-c07f-4b5a-8fe8-781fe8527536 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:57.078201 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 360/726] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:21:56 2022] POST /identity/v3/endpoints => generated 346 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:21:57.140585 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b5db6-bd5d-428a-81dd-3646fc4cb53a None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:57.248316 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:57.255079 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44b5db6-bd5d-428a-81dd-3646fc4cb53a None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:57.298135 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8c9ad5-c35f-49a1-b814-acb43bde8912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:57.298722 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8c9ad5-c35f-49a1-b814-acb43bde8912 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:57.298995 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8c9ad5-c35f-49a1-b814-acb43bde8912 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:57.299262 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8c9ad5-c35f-49a1-b814-acb43bde8912 None admin] PATH_INFO: `/v3/services/image` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:57.299836 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8c9ad5-c35f-49a1-b814-acb43bde8912 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:57.301205 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8c9ad5-c35f-49a1-b814-acb43bde8912 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:57.310539 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-9f8c9ad5-c35f-49a1-b814-acb43bde8912 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 03:21:57.310539 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:57.310539 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 03:21:57.310539 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 03:21:57.310539 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 03:21:57.310539 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 03:21:57.310539 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 03:21:57.310539 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 03:21:57.310539 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 03:21:57.310539 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:57.310539 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 03:21:57.310539 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:57.310539 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:57.310539 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:57.310539 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:57.310539 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:57.310539 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:57.311735 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:57.311735 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:57.311735 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:57.311735 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:57.311735 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:57.311735 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:57.311735 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 03:21:57.311735 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 03:21:57.311735 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 03:21:57.311735 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 03:21:57.311735 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:57.311735 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:57.311735 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:57.311735 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:57.311735 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:57.311735 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:57.312958 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:57.312958 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:57.312958 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:57.312958 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:57.312958 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:57.312958 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:57.312958 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:57.312958 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:57.312958 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:57.312958 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:57.312958 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 03:21:57.312958 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 03:21:57.312958 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 03:21:57.312958 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:21:57.312958 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 367/727] 10.222.0.26 () {56 vars in 1105 bytes} [Wed Nov 2 03:21:57 2022] GET /identity/v3/services/image => generated 86 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:57.317642 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8022db6-c07f-4b5a-8fe8-781fe8527536 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:57.321738 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:57.428837 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8022db6-c07f-4b5a-8fe8-781fe8527536 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:57.464604 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a5bd71-3303-4005-af59-468d6198c241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:57.465234 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a5bd71-3303-4005-af59-468d6198c241 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:57.465511 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a5bd71-3303-4005-af59-468d6198c241 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:57.465784 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a5bd71-3303-4005-af59-468d6198c241 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:57.466416 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a5bd71-3303-4005-af59-468d6198c241 None admin] RBAC: Adding query filter params (name=image) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:57.466717 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a5bd71-3303-4005-af59-468d6198c241 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:57.468126 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a5bd71-3303-4005-af59-468d6198c241 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:57.473570 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 361/728] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:21:57 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:57.478832 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8c9ad5-c35f-49a1-b814-acb43bde8912 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:57.589072 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8c9ad5-c35f-49a1-b814-acb43bde8912 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:57.623854 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c443f1-4ac3-4022-8d6c-fb599ed97749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:57.624537 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c443f1-4ac3-4022-8d6c-fb599ed97749 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:57.624817 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c443f1-4ac3-4022-8d6c-fb599ed97749 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:57.625092 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c443f1-4ac3-4022-8d6c-fb599ed97749 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:57.625746 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c443f1-4ac3-4022-8d6c-fb599ed97749 None admin] RBAC: Adding query filter params (type=image) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:57.626051 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c443f1-4ac3-4022-8d6c-fb599ed97749 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:57.627461 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c443f1-4ac3-4022-8d6c-fb599ed97749 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:57.633770 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 368/729] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:21:57 2022] GET /identity/v3/services?type=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:57.753109 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0892adb-634d-43e6-95b4-53d6e7ce2502 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:57.753466 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0892adb-634d-43e6-95b4-53d6e7ce2502 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:57.753665 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0892adb-634d-43e6-95b4-53d6e7ce2502 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:57.755299 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 362/730] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:21:57.761941 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87471c72-92e6-4b55-80ae-d095ef7d175d None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:57.762253 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87471c72-92e6-4b55-80ae-d095ef7d175d None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:57.762507 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87471c72-92e6-4b55-80ae-d095ef7d175d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:57.782152 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-87471c72-92e6-4b55-80ae-d095ef7d175d None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:57.783362 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-87471c72-92e6-4b55-80ae-d095ef7d175d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:57.798214 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:57.858795 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 369/731] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 03:21:58.131228 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0892adb-634d-43e6-95b4-53d6e7ce2502 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:58.210905 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:58.247184 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0892adb-634d-43e6-95b4-53d6e7ce2502 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:58.280909 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a1a5a6-e89b-4eb4-95be-51c912df1ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:58.281539 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a1a5a6-e89b-4eb4-95be-51c912df1ec9 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:58.281812 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a1a5a6-e89b-4eb4-95be-51c912df1ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:58.282082 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a1a5a6-e89b-4eb4-95be-51c912df1ec9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:58.282660 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a1a5a6-e89b-4eb4-95be-51c912df1ec9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:58.284040 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a1a5a6-e89b-4eb4-95be-51c912df1ec9 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:58.293540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-c9a1a5a6-e89b-4eb4-95be-51c912df1ec9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 03:21:58.293540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:58.293540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 03:21:58.293540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 03:21:58.293540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 03:21:58.293540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 03:21:58.293540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 03:21:58.293540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 03:21:58.293540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 03:21:58.293540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:58.293540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 03:21:58.293540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:58.293540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:58.293540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:58.293540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:58.293540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:58.293540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:58.294509 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:58.294509 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:58.294509 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:58.294509 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:58.294509 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:58.294509 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:58.294509 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 03:21:58.294509 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 03:21:58.294509 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 03:21:58.294509 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 03:21:58.294509 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:58.294509 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:58.294509 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:21:58.294509 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:21:58.294509 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:21:58.294509 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:21:58.295353 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:21:58.295353 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:21:58.295353 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:21:58.295353 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:21:58.295353 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:21:58.295353 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:21:58.295353 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:21:58.295353 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:21:58.295353 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:21:58.295353 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:21:58.295353 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 03:21:58.295353 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 03:21:58.295353 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 03:21:58.295353 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:58.295353 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 363/732] 10.222.0.26 () {56 vars in 1126 bytes} [Wed Nov 2 03:21:58 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:58.302544 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87471c72-92e6-4b55-80ae-d095ef7d175d None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:58.309823 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf00549-2dbe-4f95-9c0f-35e7751c4a0d None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:58.310102 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf00549-2dbe-4f95-9c0f-35e7751c4a0d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:58.310376 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf00549-2dbe-4f95-9c0f-35e7751c4a0d None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:58.311986 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 364/733] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 03:21:58.320301 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d065aa37-7191-47c6-a854-600e0ef48216 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:58.320873 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d065aa37-7191-47c6-a854-600e0ef48216 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:58.320873 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d065aa37-7191-47c6-a854-600e0ef48216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:58.338799 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d065aa37-7191-47c6-a854-600e0ef48216 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:58.340004 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d065aa37-7191-47c6-a854-600e0ef48216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:58.412365 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87471c72-92e6-4b55-80ae-d095ef7d175d None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:58.419008 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 365/734] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 03:21:58.448234 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d3464a-b9e3-4959-96dd-b1d79d164baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:58.449205 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3464a-b9e3-4959-96dd-b1d79d164baf None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:58.449690 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3464a-b9e3-4959-96dd-b1d79d164baf None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:58.450160 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d3464a-b9e3-4959-96dd-b1d79d164baf None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:58.451023 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d3464a-b9e3-4959-96dd-b1d79d164baf None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:58.451551 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d3464a-b9e3-4959-96dd-b1d79d164baf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:58.453179 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d3464a-b9e3-4959-96dd-b1d79d164baf None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:58.458483 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 370/735] 10.222.0.26 () {56 vars in 1122 bytes} [Wed Nov 2 03:21:58 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:58.464886 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d065aa37-7191-47c6-a854-600e0ef48216 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:58.556300 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:58.578534 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d065aa37-7191-47c6-a854-600e0ef48216 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:58.616835 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fac9c8-d725-4f90-8091-0c621f717030 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:58.617841 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fac9c8-d725-4f90-8091-0c621f717030 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:58.618341 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fac9c8-d725-4f90-8091-0c621f717030 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:58.618811 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fac9c8-d725-4f90-8091-0c621f717030 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:58.619628 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fac9c8-d725-4f90-8091-0c621f717030 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:58.620172 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fac9c8-d725-4f90-8091-0c621f717030 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:58.621801 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fac9c8-d725-4f90-8091-0c621f717030 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:58.627112 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 366/736] 10.222.0.26 () {56 vars in 1122 bytes} [Wed Nov 2 03:21:58 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:58.632818 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d3464a-b9e3-4959-96dd-b1d79d164baf None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:58.648144 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:58.704673 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fac9c8-d725-4f90-8091-0c621f717030 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:58.745588 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d3464a-b9e3-4959-96dd-b1d79d164baf None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:58.790330 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb5e88d-d416-43b3-a53d-719bfd45c978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:58.791562 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb5e88d-d416-43b3-a53d-719bfd45c978 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:58.792039 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb5e88d-d416-43b3-a53d-719bfd45c978 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:58.792509 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb5e88d-d416-43b3-a53d-719bfd45c978 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:58.794084 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb5e88d-d416-43b3-a53d-719bfd45c978 None admin] RBAC: Adding query filter params (interface=admin, service_id=6a28dd0bc30d4fab838f2c55847008bb, region_id=RegionOne) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:21:58.794573 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb5e88d-d416-43b3-a53d-719bfd45c978 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:58.796635 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb5e88d-d416-43b3-a53d-719bfd45c978 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:58.808116 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 371/737] 10.222.0.26 () {56 vars in 1249 bytes} [Wed Nov 2 03:21:58 2022] GET /identity/v3/endpoints?service_id=6a28dd0bc30d4fab838f2c55847008bb&interface=admin®ion_id=RegionOne => generated 193 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:21:58.810418 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb5e88d-d416-43b3-a53d-719bfd45c978 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:58.818007 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fac9c8-d725-4f90-8091-0c621f717030 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:58.847052 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:58.858387 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232a74e8-61d7-466c-a2f9-95e673241bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:58.859004 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232a74e8-61d7-466c-a2f9-95e673241bca None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:58.859279 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232a74e8-61d7-466c-a2f9-95e673241bca None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:58.859546 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232a74e8-61d7-466c-a2f9-95e673241bca None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:58.860226 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232a74e8-61d7-466c-a2f9-95e673241bca None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:58.861584 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232a74e8-61d7-466c-a2f9-95e673241bca None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:58.882006 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 367/738] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Nov 2 03:21:58 2022] POST /identity/v3/services => generated 242 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 03:21:58.917075 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:21:58.946462 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb5e88d-d416-43b3-a53d-719bfd45c978 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:58.996394 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf9c99a-248c-49dd-90a5-a9370f3b3eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:58.997080 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf9c99a-248c-49dd-90a5-a9370f3b3eae None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:58.997366 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf9c99a-248c-49dd-90a5-a9370f3b3eae None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:58.997642 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf9c99a-248c-49dd-90a5-a9370f3b3eae None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:58.998253 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf9c99a-248c-49dd-90a5-a9370f3b3eae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:58.999649 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf9c99a-248c-49dd-90a5-a9370f3b3eae None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:59.007663 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 372/739] 10.222.0.26 () {56 vars in 1087 bytes} [Wed Nov 2 03:21:58 2022] GET /identity/v3/services => generated 1727 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 03:21:59.392868 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e714099e-5bf9-49b5-be96-2a349960f878 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:59.393348 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e714099e-5bf9-49b5-be96-2a349960f878 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:59.393348 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e714099e-5bf9-49b5-be96-2a349960f878 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:59.394953 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 368/740] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:21:59.400935 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf5b16-d429-46ce-9d3f-8d951bdc8ef3 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:59.401502 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf5b16-d429-46ce-9d3f-8d951bdc8ef3 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:59.401979 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaf5b16-d429-46ce-9d3f-8d951bdc8ef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:59.421279 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-fdaf5b16-d429-46ce-9d3f-8d951bdc8ef3 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:59.422801 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-fdaf5b16-d429-46ce-9d3f-8d951bdc8ef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:59.493697 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 373/741] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:59 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 03:21:59.774514 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e714099e-5bf9-49b5-be96-2a349960f878 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:21:59.888452 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e714099e-5bf9-49b5-be96-2a349960f878 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:21:59.919844 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9561276d-858a-4a83-a90c-ae0e8faf8308 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:59.920391 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9561276d-858a-4a83-a90c-ae0e8faf8308 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:59.920582 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9561276d-858a-4a83-a90c-ae0e8faf8308 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:59.922003 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 374/742] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:21:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:21:59.924786 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd749fb-998f-4680-9631-f57b8a1800ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:21:59.925374 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd749fb-998f-4680-9631-f57b8a1800ee None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:59.925720 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd749fb-998f-4680-9631-f57b8a1800ee None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:59.926001 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd749fb-998f-4680-9631-f57b8a1800ee None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:59.926681 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd749fb-998f-4680-9631-f57b8a1800ee None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:21:59.928279 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd749fb-998f-4680-9631-f57b8a1800ee None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:21:59.928659 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15159ee7-fd6f-43ce-8f74-d0110233a57a None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:21:59.929140 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15159ee7-fd6f-43ce-8f74-d0110233a57a None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:21:59.929631 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15159ee7-fd6f-43ce-8f74-d0110233a57a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:21:59.931068 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: WARNING keystone.server.flask.common [None req-0dd749fb-998f-4680-9631-f57b8a1800ee None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Nov 02 03:21:59.951460 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-15159ee7-fd6f-43ce-8f74-d0110233a57a None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:21:59.952398 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.sql.core [None req-0dd749fb-998f-4680-9631-f57b8a1800ee None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Nov 02 03:21:59.952398 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Nov 02 03:21:59.952398 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [parameters: {'id': '6838ff6c9c3e4b0da25d305e8bee1033', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Nov 02 03:21:59.952398 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81503) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Nov 02 03:21:59.953639 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-15159ee7-fd6f-43ce-8f74-d0110233a57a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:21:59.960209 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-0dd749fb-998f-4680-9631-f57b8a1800ee None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6838ff6c9c3e4b0da25d305e8bee1033.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6838ff6c9c3e4b0da25d305e8bee1033. Nov 02 03:21:59.960209 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:59.960209 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 03:21:59.960209 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 03:21:59.960209 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 03:21:59.960209 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 03:21:59.960209 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 03:21:59.960209 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 03:21:59.960209 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 03:21:59.960209 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application conn.query(q) Nov 02 03:21:59.960209 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 03:21:59.960209 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 03:21:59.960209 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 03:21:59.961051 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application result.read() Nov 02 03:21:59.961051 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 03:21:59.961051 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 03:21:59.961051 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 03:21:59.961051 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 03:21:59.961051 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 03:21:59.961051 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 03:21:59.961051 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 03:21:59.961051 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 03:21:59.961051 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Nov 02 03:21:59.961051 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:59.961051 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Nov 02 03:21:59.961051 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:59.961051 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:59.961051 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Nov 02 03:21:59.961051 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return method(*args, **kwargs) Nov 02 03:21:59.961051 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Nov 02 03:21:59.961051 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return project_ref.to_dict() Nov 02 03:21:59.962010 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 03:21:59.962010 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application next(self.gen) Nov 02 03:21:59.962010 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Nov 02 03:21:59.962010 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application yield resource Nov 02 03:21:59.962010 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 03:21:59.962010 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application next(self.gen) Nov 02 03:21:59.962010 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Nov 02 03:21:59.962010 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self.session.rollback() Nov 02 03:21:59.962010 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Nov 02 03:21:59.962010 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self.force_reraise() Nov 02 03:21:59.962010 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Nov 02 03:21:59.962010 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise self.value Nov 02 03:21:59.962010 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Nov 02 03:21:59.962010 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Nov 02 03:21:59.962010 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Nov 02 03:21:59.962010 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application session.commit() Nov 02 03:21:59.962010 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Nov 02 03:21:59.962946 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Nov 02 03:21:59.962946 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Nov 02 03:21:59.962946 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._prepare_impl() Nov 02 03:21:59.962946 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Nov 02 03:21:59.962946 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self.session.flush() Nov 02 03:21:59.962946 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Nov 02 03:21:59.962946 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._flush(objects) Nov 02 03:21:59.962946 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Nov 02 03:21:59.962946 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Nov 02 03:21:59.962946 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Nov 02 03:21:59.962946 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application compat.raise_( Nov 02 03:21:59.962946 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 03:21:59.962946 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception Nov 02 03:21:59.962946 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Nov 02 03:21:59.962946 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application flush_context.execute() Nov 02 03:21:59.962946 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Nov 02 03:21:59.962946 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rec.execute(self) Nov 02 03:21:59.963848 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Nov 02 03:21:59.963848 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Nov 02 03:21:59.963848 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Nov 02 03:21:59.963848 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application _emit_insert_statements( Nov 02 03:21:59.963848 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Nov 02 03:21:59.963848 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application c = connection._execute_20( Nov 02 03:21:59.963848 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Nov 02 03:21:59.963848 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Nov 02 03:21:59.963848 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Nov 02 03:21:59.963848 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Nov 02 03:21:59.963848 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Nov 02 03:21:59.963848 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ret = self._execute_context( Nov 02 03:21:59.963848 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Nov 02 03:21:59.963848 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Nov 02 03:21:59.963848 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Nov 02 03:21:59.964673 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Nov 02 03:21:59.964673 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 03:21:59.964673 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception Nov 02 03:21:59.964673 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 03:21:59.964673 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 03:21:59.964673 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 03:21:59.964673 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 03:21:59.964673 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 03:21:59.964673 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 03:21:59.964673 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 03:21:59.964673 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application conn.query(q) Nov 02 03:21:59.964673 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 03:21:59.964673 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 03:21:59.964673 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 03:21:59.964673 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application result.read() Nov 02 03:21:59.964673 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 03:21:59.964673 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 03:21:59.965562 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 03:21:59.965562 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 03:21:59.965562 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 03:21:59.965562 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 03:21:59.965562 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 03:21:59.965562 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 03:21:59.965562 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Nov 02 03:21:59.965562 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Nov 02 03:21:59.965562 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [parameters: {'id': '6838ff6c9c3e4b0da25d305e8bee1033', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Nov 02 03:21:59.965562 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Nov 02 03:21:59.965562 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:59.965562 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 03:21:59.965562 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:59.965562 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:59.965562 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Nov 02 03:21:59.968861 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Nov 02 03:21:59.968861 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Nov 02 03:21:59.968861 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Nov 02 03:21:59.968861 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Nov 02 03:21:59.968861 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:59.968861 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 03:21:59.968861 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:59.968861 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:21:59.968861 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:21:59.968861 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:21:59.968861 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:21:59.968861 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:21:59.968861 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:21:59.968861 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:21:59.968861 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:21:59.968861 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:21:59.968861 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:21:59.969767 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:21:59.969767 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Nov 02 03:21:59.969767 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Nov 02 03:21:59.969767 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:21:59.969767 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:21:59.969767 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Nov 02 03:21:59.969767 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.Conflict( Nov 02 03:21:59.969767 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 6838ff6c9c3e4b0da25d305e8bee1033. Nov 02 03:21:59.969767 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:21:59.969767 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 369/743] 10.222.0.26 () {60 vars in 1140 bytes} [Wed Nov 2 03:21:59 2022] POST /identity/v3/projects => generated 262 bytes in 188 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:21:59.972908 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd749fb-998f-4680-9631-f57b8a1800ee None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:00.022287 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:00.039677 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:00.056292 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 375/744] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:21:59 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 03:22:00.095515 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd749fb-998f-4680-9631-f57b8a1800ee None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:00.130494 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3ecc03-1352-4a21-b37d-ad578cc489b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:00.131202 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ecc03-1352-4a21-b37d-ad578cc489b5 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:00.131484 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ecc03-1352-4a21-b37d-ad578cc489b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:00.131757 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3ecc03-1352-4a21-b37d-ad578cc489b5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:00.138759 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3ecc03-1352-4a21-b37d-ad578cc489b5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:00.140491 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3ecc03-1352-4a21-b37d-ad578cc489b5 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:00.152549 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-db3ecc03-1352-4a21-b37d-ad578cc489b5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 03:22:00.152549 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:22:00.152549 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:22:00.152549 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:22:00.152549 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:22:00.152549 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:22:00.152549 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:22:00.152549 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:22:00.152549 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:22:00.152549 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:22:00.152549 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:22:00.152549 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:22:00.152549 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 03:22:00.152549 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 03:22:00.152549 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 03:22:00.153448 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 03:22:00.153448 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:22:00.153448 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:22:00.153448 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:22:00.153448 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:22:00.153448 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:22:00.153448 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:22:00.153448 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:22:00.153448 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:22:00.153448 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:22:00.153448 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:22:00.153448 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:22:00.153448 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:22:00.153448 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:22:00.153448 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:22:00.153448 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:22:00.153448 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:22:00.154344 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 03:22:00.154344 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 03:22:00.154344 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 03:22:00.154344 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 03:22:00.154344 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 03:22:00.154344 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 03:22:00.154344 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 03:22:00.154344 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:22:00.154344 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 370/745] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Nov 2 03:21:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:22:00.159666 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15159ee7-fd6f-43ce-8f74-d0110233a57a None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:00.268614 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15159ee7-fd6f-43ce-8f74-d0110233a57a None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:00.309036 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbcffe0-a819-45e6-b35c-b55b638da489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:00.309579 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbcffe0-a819-45e6-b35c-b55b638da489 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:00.309861 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbcffe0-a819-45e6-b35c-b55b638da489 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:00.310138 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbcffe0-a819-45e6-b35c-b55b638da489 None admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:00.310863 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbcffe0-a819-45e6-b35c-b55b638da489 None admin] RBAC: Adding query filter params (name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:22:00.311170 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbcffe0-a819-45e6-b35c-b55b638da489 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:00.312627 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbcffe0-a819-45e6-b35c-b55b638da489 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:00.330436 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 376/746] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Nov 2 03:22:00 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:22:00.361459 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3ecc03-1352-4a21-b37d-ad578cc489b5 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:00.474974 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3ecc03-1352-4a21-b37d-ad578cc489b5 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:00.482853 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:00.512430 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a96324-5b98-4c2d-b667-ddfa99470f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:00.513050 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a96324-5b98-4c2d-b667-ddfa99470f3d None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:00.513328 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a96324-5b98-4c2d-b667-ddfa99470f3d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:00.513619 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a96324-5b98-4c2d-b667-ddfa99470f3d None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:00.514266 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a96324-5b98-4c2d-b667-ddfa99470f3d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:00.515740 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a96324-5b98-4c2d-b667-ddfa99470f3d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:00.542849 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 371/747] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Nov 2 03:22:00 2022] POST /identity/v3/services => generated 242 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:22:01.110226 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a8a8f0-6ae3-4c4a-99ce-10460e731a5e None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:01.110764 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a8a8f0-6ae3-4c4a-99ce-10460e731a5e None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:01.110764 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a8a8f0-6ae3-4c4a-99ce-10460e731a5e None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:01.112818 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 377/748] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:22:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:22:01.119127 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7639401c-b704-4e0c-b3b0-99e083f1d03f None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:01.119449 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7639401c-b704-4e0c-b3b0-99e083f1d03f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:01.119682 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7639401c-b704-4e0c-b3b0-99e083f1d03f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:01.139641 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-7639401c-b704-4e0c-b3b0-99e083f1d03f None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:22:01.141195 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7639401c-b704-4e0c-b3b0-99e083f1d03f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:22:01.218144 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 372/749] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:22:01 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 03:22:01.227234 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1292338a-0c66-4056-a4b8-39e24d8c96db None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:01.227523 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1292338a-0c66-4056-a4b8-39e24d8c96db None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:01.227799 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1292338a-0c66-4056-a4b8-39e24d8c96db None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:01.229418 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 378/750] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:22:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:22:01.237336 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5911b-e57e-4a70-a912-065b0a9f8d5c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:01.237882 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5911b-e57e-4a70-a912-065b0a9f8d5c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:01.238152 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5911b-e57e-4a70-a912-065b0a9f8d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:01.257170 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-57e5911b-e57e-4a70-a912-065b0a9f8d5c None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:22:01.258353 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-57e5911b-e57e-4a70-a912-065b0a9f8d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:22:01.326446 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 373/751] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:22:01 2022] POST /identity/v3/auth/tokens => generated 1666 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 03:22:01.488805 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1292338a-0c66-4056-a4b8-39e24d8c96db None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:01.546621 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:01.592415 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1292338a-0c66-4056-a4b8-39e24d8c96db None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:01.596043 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e5911b-e57e-4a70-a912-065b0a9f8d5c None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:01.620690 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:01.633709 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30882ad4-4d07-42de-8cf3-5a87c942afb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:01.634904 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30882ad4-4d07-42de-8cf3-5a87c942afb2 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:01.635441 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30882ad4-4d07-42de-8cf3-5a87c942afb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:01.636083 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30882ad4-4d07-42de-8cf3-5a87c942afb2 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:01.638077 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30882ad4-4d07-42de-8cf3-5a87c942afb2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:01.639816 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30882ad4-4d07-42de-8cf3-5a87c942afb2 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:01.655644 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-30882ad4-4d07-42de-8cf3-5a87c942afb2 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 03:22:01.655644 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:22:01.655644 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 03:22:01.655644 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 03:22:01.655644 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 03:22:01.655644 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 03:22:01.655644 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 03:22:01.655644 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 03:22:01.655644 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 03:22:01.655644 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:22:01.655644 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 03:22:01.655644 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:22:01.655644 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:22:01.655644 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:22:01.655644 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:22:01.655644 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:22:01.655644 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:22:01.656712 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:22:01.656712 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:22:01.656712 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:22:01.656712 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:22:01.656712 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:22:01.656712 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:22:01.656712 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 03:22:01.656712 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 03:22:01.656712 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 03:22:01.656712 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 03:22:01.656712 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:22:01.656712 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:22:01.656712 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:22:01.656712 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:22:01.656712 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:22:01.656712 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:22:01.657550 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:22:01.657550 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:22:01.657550 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:22:01.657550 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:22:01.657550 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:22:01.657550 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:22:01.657550 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:22:01.657550 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:22:01.657550 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:22:01.657550 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:22:01.657550 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 03:22:01.657550 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 03:22:01.657550 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 03:22:01.657550 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:22:01.660596 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 379/752] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Nov 2 03:22:01 2022] GET /identity/v3/services/compute => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:22:01.668985 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30882ad4-4d07-42de-8cf3-5a87c942afb2 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:01.719499 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e5911b-e57e-4a70-a912-065b0a9f8d5c None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:01.757612 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514f7af1-c583-4d7f-8984-0963f3e0bf5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:01.758358 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f7af1-c583-4d7f-8984-0963f3e0bf5e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:01.758794 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f7af1-c583-4d7f-8984-0963f3e0bf5e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:01.759136 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514f7af1-c583-4d7f-8984-0963f3e0bf5e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:01.759783 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514f7af1-c583-4d7f-8984-0963f3e0bf5e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:01.761252 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514f7af1-c583-4d7f-8984-0963f3e0bf5e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:01.773746 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-514f7af1-c583-4d7f-8984-0963f3e0bf5e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 03:22:01.773746 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:22:01.773746 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 03:22:01.773746 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 03:22:01.773746 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 03:22:01.773746 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 03:22:01.773746 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 03:22:01.773746 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 03:22:01.773746 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 03:22:01.773746 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:22:01.773746 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 03:22:01.773746 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:22:01.773746 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:22:01.773746 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:22:01.773746 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:22:01.773746 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:22:01.773746 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:22:01.774962 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:22:01.774962 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:22:01.774962 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:22:01.774962 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:22:01.774962 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:22:01.774962 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:22:01.774962 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 03:22:01.774962 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 03:22:01.774962 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 03:22:01.774962 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 03:22:01.774962 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:22:01.774962 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:22:01.774962 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:22:01.774962 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:22:01.774962 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:22:01.774962 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:22:01.776049 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:22:01.776049 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:22:01.776049 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:22:01.776049 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:22:01.776049 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:22:01.776049 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:22:01.776049 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:22:01.776049 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:22:01.776049 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:22:01.776049 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:22:01.776049 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 03:22:01.776049 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 03:22:01.776049 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Nov 02 03:22:01.776049 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:22:01.779992 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 374/753] 10.222.0.26 () {56 vars in 1126 bytes} [Wed Nov 2 03:22:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 03:22:01.782643 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514f7af1-c583-4d7f-8984-0963f3e0bf5e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:01.801675 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30882ad4-4d07-42de-8cf3-5a87c942afb2 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:01.830057 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:01.849774 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679ec05d-f0d9-4054-b7eb-7037e1bba368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:01.851241 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ec05d-f0d9-4054-b7eb-7037e1bba368 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:01.851768 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ec05d-f0d9-4054-b7eb-7037e1bba368 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:01.852254 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679ec05d-f0d9-4054-b7eb-7037e1bba368 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:01.853105 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679ec05d-f0d9-4054-b7eb-7037e1bba368 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:22:01.853739 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679ec05d-f0d9-4054-b7eb-7037e1bba368 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:01.855772 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679ec05d-f0d9-4054-b7eb-7037e1bba368 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:01.867504 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 380/754] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Nov 2 03:22:01 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:22:01.868762 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679ec05d-f0d9-4054-b7eb-7037e1bba368 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:01.879596 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:01.905111 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514f7af1-c583-4d7f-8984-0963f3e0bf5e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:01.940802 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4073cb-0843-4efa-8133-a8dc1df37317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:01.941436 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4073cb-0843-4efa-8133-a8dc1df37317 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:01.941711 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4073cb-0843-4efa-8133-a8dc1df37317 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:01.941978 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4073cb-0843-4efa-8133-a8dc1df37317 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:01.942595 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4073cb-0843-4efa-8133-a8dc1df37317 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:22:01.942891 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4073cb-0843-4efa-8133-a8dc1df37317 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:01.945142 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4073cb-0843-4efa-8133-a8dc1df37317 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:01.954819 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 375/755] 10.222.0.26 () {56 vars in 1122 bytes} [Wed Nov 2 03:22:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:22:01.956138 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4073cb-0843-4efa-8133-a8dc1df37317 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:02.010294 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679ec05d-f0d9-4054-b7eb-7037e1bba368 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:02.057409 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd6bec5-dcc1-4377-b784-b84b4aaecadc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:02.058609 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6bec5-dcc1-4377-b784-b84b4aaecadc None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:02.059090 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6bec5-dcc1-4377-b784-b84b4aaecadc None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:02.059563 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd6bec5-dcc1-4377-b784-b84b4aaecadc None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:02.059870 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4073cb-0843-4efa-8133-a8dc1df37317 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:02.060646 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd6bec5-dcc1-4377-b784-b84b4aaecadc None admin] RBAC: Adding query filter params (type=compute) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:22:02.061176 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd6bec5-dcc1-4377-b784-b84b4aaecadc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:02.063782 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd6bec5-dcc1-4377-b784-b84b4aaecadc None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:02.075624 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 381/756] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Nov 2 03:22:01 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:22:02.077033 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd6bec5-dcc1-4377-b784-b84b4aaecadc None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:02.102565 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad8fde-6be1-4d94-9fc6-9f7c38758d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:02.103093 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad8fde-6be1-4d94-9fc6-9f7c38758d2b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:02.103368 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad8fde-6be1-4d94-9fc6-9f7c38758d2b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:02.103637 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad8fde-6be1-4d94-9fc6-9f7c38758d2b None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:02.104463 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aad8fde-6be1-4d94-9fc6-9f7c38758d2b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:22:02.104751 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aad8fde-6be1-4d94-9fc6-9f7c38758d2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:02.106161 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aad8fde-6be1-4d94-9fc6-9f7c38758d2b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:02.116130 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 376/757] 10.222.0.26 () {56 vars in 1122 bytes} [Wed Nov 2 03:22:01 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:22:02.118331 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad8fde-6be1-4d94-9fc6-9f7c38758d2b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:02.132077 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:02.144326 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:02.217008 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd6bec5-dcc1-4377-b784-b84b4aaecadc None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:02.241053 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad8fde-6be1-4d94-9fc6-9f7c38758d2b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:02.260176 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef3bb7d-beb2-4618-88f2-95b5ef49589c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:02.260897 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef3bb7d-beb2-4618-88f2-95b5ef49589c None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:02.261077 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef3bb7d-beb2-4618-88f2-95b5ef49589c None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:02.261316 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef3bb7d-beb2-4618-88f2-95b5ef49589c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:02.262068 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef3bb7d-beb2-4618-88f2-95b5ef49589c None admin] RBAC: Adding query filter params (interface=public, service_id=9eb3451db9ed4c64932c53c14805e689, region_id=RegionOne) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:22:02.262375 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef3bb7d-beb2-4618-88f2-95b5ef49589c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:02.263738 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef3bb7d-beb2-4618-88f2-95b5ef49589c None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:02.271982 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 382/758] 10.222.0.26 () {56 vars in 1251 bytes} [Wed Nov 2 03:22:02 2022] GET /identity/v3/endpoints?service_id=9eb3451db9ed4c64932c53c14805e689&interface=public®ion_id=RegionOne => generated 194 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:22:02.275338 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef3bb7d-beb2-4618-88f2-95b5ef49589c None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:02.293442 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9754f34-fc33-425e-bb3e-564d71fdd590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:02.294403 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9754f34-fc33-425e-bb3e-564d71fdd590 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:02.295119 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9754f34-fc33-425e-bb3e-564d71fdd590 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:02.295939 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9754f34-fc33-425e-bb3e-564d71fdd590 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:02.296814 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9754f34-fc33-425e-bb3e-564d71fdd590 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:02.298380 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9754f34-fc33-425e-bb3e-564d71fdd590 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:02.323082 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 377/759] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:22:02 2022] POST /identity/v3/endpoints => generated 325 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 03:22:02.405642 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef3bb7d-beb2-4618-88f2-95b5ef49589c None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:02.422244 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:02.455574 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ee0ef7-3374-48f8-9dfe-2637e5e10f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:02.456237 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee0ef7-3374-48f8-9dfe-2637e5e10f32 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:02.456514 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee0ef7-3374-48f8-9dfe-2637e5e10f32 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:02.456784 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ee0ef7-3374-48f8-9dfe-2637e5e10f32 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:02.457383 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ee0ef7-3374-48f8-9dfe-2637e5e10f32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:02.458770 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ee0ef7-3374-48f8-9dfe-2637e5e10f32 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:02.467034 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 383/760] 10.222.0.26 () {56 vars in 1087 bytes} [Wed Nov 2 03:22:02 2022] GET /identity/v3/services => generated 1957 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 03:22:02.725662 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5101980b-0edf-489a-8563-519f077516f5 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:02.726194 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5101980b-0edf-489a-8563-519f077516f5 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:02.726194 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5101980b-0edf-489a-8563-519f077516f5 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:02.727795 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 378/761] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:22:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:22:02.733083 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd30473-f80d-4ae3-aaa8-b6481cbce631 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:02.733370 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd30473-f80d-4ae3-aaa8-b6481cbce631 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:02.733648 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd30473-f80d-4ae3-aaa8-b6481cbce631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:02.753224 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-edd30473-f80d-4ae3-aaa8-b6481cbce631 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:22:02.754416 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-edd30473-f80d-4ae3-aaa8-b6481cbce631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:22:02.790413 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d6eeb-1a3b-42b2-bfb3-7ccc46f6a0c8 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:02.790619 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d6eeb-1a3b-42b2-bfb3-7ccc46f6a0c8 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:02.790890 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381d6eeb-1a3b-42b2-bfb3-7ccc46f6a0c8 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:02.792414 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 379/762] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:22:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 03:22:02.797249 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abe4c8d-a068-47d5-90c7-fbdd38b8397e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:02.797535 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abe4c8d-a068-47d5-90c7-fbdd38b8397e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:02.797812 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abe4c8d-a068-47d5-90c7-fbdd38b8397e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:02.816797 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-2abe4c8d-a068-47d5-90c7-fbdd38b8397e None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:22:02.818207 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2abe4c8d-a068-47d5-90c7-fbdd38b8397e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:22:02.826646 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 384/763] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:22:02 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 03:22:02.888314 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:02.894005 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 380/764] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:22:02 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 03:22:03.107077 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd30473-f80d-4ae3-aaa8-b6481cbce631 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:03.166218 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abe4c8d-a068-47d5-90c7-fbdd38b8397e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:03.209644 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd30473-f80d-4ae3-aaa8-b6481cbce631 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:03.211220 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:03.251729 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf37be55-ffb2-4018-af4b-14a6dae4a87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:03.252379 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37be55-ffb2-4018-af4b-14a6dae4a87b None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:03.252660 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37be55-ffb2-4018-af4b-14a6dae4a87b None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:03.252935 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf37be55-ffb2-4018-af4b-14a6dae4a87b None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:03.253590 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf37be55-ffb2-4018-af4b-14a6dae4a87b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:03.255028 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf37be55-ffb2-4018-af4b-14a6dae4a87b None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:03.258763 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: WARNING keystone.server.flask.common [None req-cf37be55-ffb2-4018-af4b-14a6dae4a87b None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Nov 02 03:22:03.276056 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.sql.core [None req-cf37be55-ffb2-4018-af4b-14a6dae4a87b None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Nov 02 03:22:03.276056 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Nov 02 03:22:03.276056 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [parameters: {'user_id': '9f96d57c56194f43a2a3db0a58c5b9fe', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Nov 02 03:22:03.276056 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81504) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Nov 02 03:22:03.283915 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abe4c8d-a068-47d5-90c7-fbdd38b8397e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:03.285379 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-cf37be55-ffb2-4018-af4b-14a6dae4a87b None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Nov 02 03:22:03.285379 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:22:03.285379 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 03:22:03.285379 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 03:22:03.285379 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 03:22:03.285379 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 03:22:03.285379 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 03:22:03.285379 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 03:22:03.285379 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 03:22:03.285379 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application conn.query(q) Nov 02 03:22:03.285379 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 03:22:03.285379 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 03:22:03.285379 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 03:22:03.285379 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application result.read() Nov 02 03:22:03.285379 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 03:22:03.286229 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 03:22:03.286229 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 03:22:03.286229 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 03:22:03.286229 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 03:22:03.286229 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 03:22:03.286229 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 03:22:03.286229 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 03:22:03.286229 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Nov 02 03:22:03.286229 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:22:03.286229 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Nov 02 03:22:03.286229 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:22:03.286229 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:22:03.286229 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Nov 02 03:22:03.286229 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return method(*args, **kwargs) Nov 02 03:22:03.286229 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Nov 02 03:22:03.286229 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Nov 02 03:22:03.286229 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 03:22:03.286229 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application next(self.gen) Nov 02 03:22:03.287257 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Nov 02 03:22:03.287257 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application yield resource Nov 02 03:22:03.287257 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 03:22:03.287257 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application next(self.gen) Nov 02 03:22:03.287257 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Nov 02 03:22:03.287257 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self.session.rollback() Nov 02 03:22:03.287257 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Nov 02 03:22:03.287257 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self.force_reraise() Nov 02 03:22:03.287257 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Nov 02 03:22:03.287257 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise self.value Nov 02 03:22:03.287257 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Nov 02 03:22:03.287257 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Nov 02 03:22:03.287257 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Nov 02 03:22:03.287257 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application session.commit() Nov 02 03:22:03.287257 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Nov 02 03:22:03.287257 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Nov 02 03:22:03.287257 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Nov 02 03:22:03.288160 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._prepare_impl() Nov 02 03:22:03.288160 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Nov 02 03:22:03.288160 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self.session.flush() Nov 02 03:22:03.288160 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Nov 02 03:22:03.288160 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._flush(objects) Nov 02 03:22:03.288160 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Nov 02 03:22:03.288160 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Nov 02 03:22:03.288160 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Nov 02 03:22:03.288160 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application compat.raise_( Nov 02 03:22:03.288160 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 03:22:03.288160 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception Nov 02 03:22:03.288160 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Nov 02 03:22:03.288160 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application flush_context.execute() Nov 02 03:22:03.288160 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Nov 02 03:22:03.288160 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rec.execute(self) Nov 02 03:22:03.288160 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Nov 02 03:22:03.288160 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Nov 02 03:22:03.289065 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Nov 02 03:22:03.289065 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application _emit_insert_statements( Nov 02 03:22:03.289065 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Nov 02 03:22:03.289065 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application result = connection._execute_20( Nov 02 03:22:03.289065 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Nov 02 03:22:03.289065 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Nov 02 03:22:03.289065 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Nov 02 03:22:03.289065 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Nov 02 03:22:03.289065 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Nov 02 03:22:03.289065 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ret = self._execute_context( Nov 02 03:22:03.289065 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Nov 02 03:22:03.289065 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Nov 02 03:22:03.289065 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Nov 02 03:22:03.289065 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Nov 02 03:22:03.289065 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 03:22:03.289065 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception Nov 02 03:22:03.289903 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 03:22:03.289903 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 03:22:03.289903 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 03:22:03.289903 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 03:22:03.289903 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 03:22:03.289903 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 03:22:03.289903 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 03:22:03.289903 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application conn.query(q) Nov 02 03:22:03.289903 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 03:22:03.289903 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 03:22:03.289903 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 03:22:03.289903 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application result.read() Nov 02 03:22:03.289903 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 03:22:03.289903 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 03:22:03.289903 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 03:22:03.289903 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 03:22:03.290717 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 03:22:03.290717 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 03:22:03.290717 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 03:22:03.290717 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 03:22:03.290717 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Nov 02 03:22:03.290717 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Nov 02 03:22:03.290717 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [parameters: {'user_id': '9f96d57c56194f43a2a3db0a58c5b9fe', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Nov 02 03:22:03.290717 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Nov 02 03:22:03.290717 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:22:03.290717 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 03:22:03.290717 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:22:03.290717 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:22:03.290717 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:22:03.290717 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:22:03.290717 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:22:03.291492 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:22:03.291492 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:22:03.291492 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:22:03.291492 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:22:03.291492 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:22:03.291492 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:22:03.291492 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:22:03.291492 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Nov 02 03:22:03.291492 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Nov 02 03:22:03.291492 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:22:03.291492 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:22:03.291492 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 03:22:03.291492 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 03:22:03.291492 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 03:22:03.291492 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 03:22:03.291492 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Nov 02 03:22:03.291492 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Nov 02 03:22:03.297888 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Nov 02 03:22:03.297888 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Nov 02 03:22:03.297888 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Nov 02 03:22:03.297888 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Nov 02 03:22:03.297888 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Nov 02 03:22:03.297888 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:22:03.297888 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 385/765] 10.222.0.26 () {60 vars in 1131 bytes} [Wed Nov 2 03:22:03 2022] POST /identity/v3/users => generated 160 bytes in 187 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:22:03.297888 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf37be55-ffb2-4018-af4b-14a6dae4a87b None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:03.339721 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12e5cf6-a54e-40d1-bde5-b03cb1435f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:03.341049 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e5cf6-a54e-40d1-bde5-b03cb1435f4b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:03.342400 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e5cf6-a54e-40d1-bde5-b03cb1435f4b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:03.342825 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12e5cf6-a54e-40d1-bde5-b03cb1435f4b None admin] PATH_INFO: `/v3/services/image` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:03.343549 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12e5cf6-a54e-40d1-bde5-b03cb1435f4b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:03.345153 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12e5cf6-a54e-40d1-bde5-b03cb1435f4b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:03.357588 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-b12e5cf6-a54e-40d1-bde5-b03cb1435f4b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 03:22:03.357588 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:22:03.357588 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 03:22:03.357588 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 03:22:03.357588 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 03:22:03.357588 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 03:22:03.357588 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 03:22:03.357588 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 03:22:03.357588 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 03:22:03.357588 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:22:03.357588 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 03:22:03.357588 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:22:03.357588 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:22:03.357588 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:22:03.357588 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:22:03.357588 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:22:03.357588 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:22:03.358789 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:22:03.358789 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:22:03.358789 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:22:03.358789 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:22:03.358789 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:22:03.358789 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:22:03.358789 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 03:22:03.358789 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 03:22:03.358789 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 03:22:03.358789 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 03:22:03.358789 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:22:03.358789 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:22:03.358789 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:22:03.358789 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:22:03.358789 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:22:03.358789 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:22:03.359802 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:22:03.359802 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:22:03.359802 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:22:03.359802 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:22:03.359802 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:22:03.359802 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:22:03.359802 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:22:03.359802 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:22:03.359802 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:22:03.359802 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:22:03.359802 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 03:22:03.359802 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 03:22:03.359802 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 03:22:03.359802 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:22:03.361403 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 381/766] 10.222.0.26 () {56 vars in 1105 bytes} [Wed Nov 2 03:22:03 2022] GET /identity/v3/services/image => generated 86 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:22:03.370855 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12e5cf6-a54e-40d1-bde5-b03cb1435f4b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:03.438331 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf37be55-ffb2-4018-af4b-14a6dae4a87b None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:03.439016 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:03.477863 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9403b9a9-1d6b-468e-8082-81335a823c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:03.479015 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9403b9a9-1d6b-468e-8082-81335a823c89 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:03.479525 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9403b9a9-1d6b-468e-8082-81335a823c89 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:03.480034 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9403b9a9-1d6b-468e-8082-81335a823c89 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:03.481805 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12e5cf6-a54e-40d1-bde5-b03cb1435f4b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:03.488837 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9403b9a9-1d6b-468e-8082-81335a823c89 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:03.491342 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9403b9a9-1d6b-468e-8082-81335a823c89 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:03.504131 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-9403b9a9-1d6b-468e-8082-81335a823c89 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Nov 02 03:22:03.504131 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:22:03.504131 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:22:03.504131 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:22:03.504131 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:22:03.504131 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:22:03.504131 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:22:03.504131 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:22:03.504131 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:22:03.504131 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:22:03.504131 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:22:03.504131 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:22:03.504131 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 03:22:03.504131 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 03:22:03.504131 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 03:22:03.504131 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 03:22:03.505129 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:22:03.505129 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:22:03.505129 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 03:22:03.505129 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 03:22:03.505129 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 03:22:03.505129 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 03:22:03.505129 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:22:03.505129 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:22:03.505129 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:22:03.505129 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:22:03.505129 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:22:03.505129 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:22:03.505129 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:22:03.505129 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:22:03.505129 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:22:03.505129 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:22:03.505129 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:22:03.505129 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:22:03.506142 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:22:03.506142 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:22:03.506142 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 03:22:03.506142 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 03:22:03.506142 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 03:22:03.506142 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 03:22:03.506142 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 03:22:03.506142 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 03:22:03.506142 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Nov 02 03:22:03.506142 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:22:03.508881 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 386/767] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Nov 2 03:22:03 2022] GET /identity/v3/users/cinder => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:22:03.514261 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9403b9a9-1d6b-468e-8082-81335a823c89 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:03.531506 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409f9443-1558-4b33-9e33-84d6a320776e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:03.531506 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f9443-1558-4b33-9e33-84d6a320776e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:03.531506 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f9443-1558-4b33-9e33-84d6a320776e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:03.531506 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409f9443-1558-4b33-9e33-84d6a320776e None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:03.531894 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409f9443-1558-4b33-9e33-84d6a320776e None admin] RBAC: Adding query filter params (name=image) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:22:03.531894 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409f9443-1558-4b33-9e33-84d6a320776e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:03.532331 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409f9443-1558-4b33-9e33-84d6a320776e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:03.542714 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 382/768] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:22:03 2022] GET /identity/v3/services?name=image => generated 122 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:22:03.543597 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409f9443-1558-4b33-9e33-84d6a320776e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:03.554022 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:03.649408 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409f9443-1558-4b33-9e33-84d6a320776e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:03.656322 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9403b9a9-1d6b-468e-8082-81335a823c89 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:03.678254 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:03.687478 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b26d8a-baed-4038-ae0f-33fc0c85d659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:03.688132 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b26d8a-baed-4038-ae0f-33fc0c85d659 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:03.688455 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b26d8a-baed-4038-ae0f-33fc0c85d659 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:03.688735 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b26d8a-baed-4038-ae0f-33fc0c85d659 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:03.689362 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b26d8a-baed-4038-ae0f-33fc0c85d659 None admin] RBAC: Adding query filter params (type=image) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:22:03.689666 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b26d8a-baed-4038-ae0f-33fc0c85d659 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:03.691026 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b26d8a-baed-4038-ae0f-33fc0c85d659 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:03.701626 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 383/769] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:22:03 2022] GET /identity/v3/services?type=image => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:22:03.702390 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b26d8a-baed-4038-ae0f-33fc0c85d659 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:03.703836 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33bb0f3-882d-474b-9467-72b6e1cb673c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:03.704873 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33bb0f3-882d-474b-9467-72b6e1cb673c None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:03.705617 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33bb0f3-882d-474b-9467-72b6e1cb673c None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:03.707052 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33bb0f3-882d-474b-9467-72b6e1cb673c None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:03.707991 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33bb0f3-882d-474b-9467-72b6e1cb673c None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:22:03.708495 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33bb0f3-882d-474b-9467-72b6e1cb673c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:03.710174 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33bb0f3-882d-474b-9467-72b6e1cb673c None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:03.762365 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 387/770] 10.222.0.26 () {56 vars in 1101 bytes} [Wed Nov 2 03:22:03 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:22:03.821250 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b26d8a-baed-4038-ae0f-33fc0c85d659 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:03.859770 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c36a4a-e13d-4e70-876c-82743d789e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:03.860424 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c36a4a-e13d-4e70-876c-82743d789e07 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:03.860818 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c36a4a-e13d-4e70-876c-82743d789e07 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:03.861094 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c36a4a-e13d-4e70-876c-82743d789e07 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:03.861797 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c36a4a-e13d-4e70-876c-82743d789e07 None admin] RBAC: Adding query filter params (interface=public, service_id=6008de6871ab4c698873d05b86a9a5f6, region_id=RegionOne) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:22:03.862102 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c36a4a-e13d-4e70-876c-82743d789e07 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:03.863737 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c36a4a-e13d-4e70-876c-82743d789e07 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:03.870100 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 384/771] 10.222.0.26 () {56 vars in 1251 bytes} [Wed Nov 2 03:22:03 2022] GET /identity/v3/endpoints?service_id=6008de6871ab4c698873d05b86a9a5f6&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:22:03.875516 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33bb0f3-882d-474b-9467-72b6e1cb673c None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:03.966356 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:03.987356 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33bb0f3-882d-474b-9467-72b6e1cb673c None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:04.029745 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d9df56-e220-4eee-a1f3-33dfeb7b3314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:04.030606 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d9df56-e220-4eee-a1f3-33dfeb7b3314 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:04.031073 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d9df56-e220-4eee-a1f3-33dfeb7b3314 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:04.031499 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d9df56-e220-4eee-a1f3-33dfeb7b3314 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:04.032339 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d9df56-e220-4eee-a1f3-33dfeb7b3314 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:04.033857 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d9df56-e220-4eee-a1f3-33dfeb7b3314 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:04.042319 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 388/772] 10.222.0.26 () {56 vars in 1087 bytes} [Wed Nov 2 03:22:03 2022] GET /identity/v3/services => generated 1957 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:22:04.608202 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27138679-fe7e-4a17-86c5-a820e4fd5da5 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:04.608564 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27138679-fe7e-4a17-86c5-a820e4fd5da5 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:04.608758 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27138679-fe7e-4a17-86c5-a820e4fd5da5 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:04.610248 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 385/773] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:22:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:22:04.616330 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568186cc-2876-4594-8266-34089cc22679 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:04.616820 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568186cc-2876-4594-8266-34089cc22679 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:04.616896 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568186cc-2876-4594-8266-34089cc22679 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:04.636299 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-568186cc-2876-4594-8266-34089cc22679 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:22:04.637883 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-568186cc-2876-4594-8266-34089cc22679 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:22:04.711543 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 389/774] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:22:04 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 03:22:04.745855 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111929fd-3466-4a89-a1cb-2cc0b1e185c4 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:04.746182 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111929fd-3466-4a89-a1cb-2cc0b1e185c4 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:04.746399 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111929fd-3466-4a89-a1cb-2cc0b1e185c4 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:04.747871 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 386/775] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:22:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:22:04.753013 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf66e5-541d-41bf-88eb-bae6dd9b2898 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:04.753305 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf66e5-541d-41bf-88eb-bae6dd9b2898 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:04.753580 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bf66e5-541d-41bf-88eb-bae6dd9b2898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:04.772302 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-e3bf66e5-541d-41bf-88eb-bae6dd9b2898 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:22:04.773479 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e3bf66e5-541d-41bf-88eb-bae6dd9b2898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:22:04.839735 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 390/776] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:22:04 2022] POST /identity/v3/auth/tokens => generated 1813 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 03:22:04.977684 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111929fd-3466-4a89-a1cb-2cc0b1e185c4 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:05.035530 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:05.081247 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111929fd-3466-4a89-a1cb-2cc0b1e185c4 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:05.111629 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bf66e5-541d-41bf-88eb-bae6dd9b2898 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:05.114015 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237bb103-53b8-44b9-bb70-7ec292f136c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:05.114640 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237bb103-53b8-44b9-bb70-7ec292f136c6 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:05.114915 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237bb103-53b8-44b9-bb70-7ec292f136c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:05.115280 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237bb103-53b8-44b9-bb70-7ec292f136c6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:05.116375 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237bb103-53b8-44b9-bb70-7ec292f136c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:05.117816 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237bb103-53b8-44b9-bb70-7ec292f136c6 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:05.119393 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 387/777] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:22:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:22:05.122058 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:05.126136 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237bb103-53b8-44b9-bb70-7ec292f136c6 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:05.222129 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bf66e5-541d-41bf-88eb-bae6dd9b2898 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:05.233034 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237bb103-53b8-44b9-bb70-7ec292f136c6 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:05.260075 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:05.262163 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48688ce-19e1-4b8c-810b-6b2a31f68513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:05.262856 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48688ce-19e1-4b8c-810b-6b2a31f68513 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:05.263066 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48688ce-19e1-4b8c-810b-6b2a31f68513 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:05.263335 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48688ce-19e1-4b8c-810b-6b2a31f68513 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:05.263913 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48688ce-19e1-4b8c-810b-6b2a31f68513 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:05.265303 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48688ce-19e1-4b8c-810b-6b2a31f68513 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:05.272009 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b79d9c2-82f8-4aed-8b95-9ba38287a35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:05.272621 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b79d9c2-82f8-4aed-8b95-9ba38287a35a None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:05.272898 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b79d9c2-82f8-4aed-8b95-9ba38287a35a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:05.273167 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b79d9c2-82f8-4aed-8b95-9ba38287a35a None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:05.273809 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b79d9c2-82f8-4aed-8b95-9ba38287a35a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:05.274541 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-b48688ce-19e1-4b8c-810b-6b2a31f68513 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 03:22:05.274541 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:22:05.274541 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 03:22:05.274541 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 03:22:05.274541 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 03:22:05.274541 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 03:22:05.274541 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 03:22:05.274541 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 03:22:05.274541 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 03:22:05.274541 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:22:05.274541 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 03:22:05.274541 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:22:05.274541 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:22:05.274541 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:22:05.274541 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:22:05.274541 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:22:05.274541 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:22:05.275481 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:22:05.275481 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:22:05.275481 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:22:05.275481 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:22:05.275481 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:22:05.275481 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:22:05.275481 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 03:22:05.275481 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 03:22:05.275481 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 03:22:05.275481 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 03:22:05.275481 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:22:05.275481 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:22:05.275481 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:22:05.275481 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:22:05.275481 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:22:05.275481 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:22:05.276444 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:22:05.276444 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:22:05.276444 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:22:05.276444 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:22:05.276444 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:22:05.276444 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:22:05.276444 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:22:05.276444 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:22:05.276444 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:22:05.276444 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:22:05.276444 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 03:22:05.276444 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 03:22:05.276444 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Nov 02 03:22:05.276444 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:22:05.277174 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b79d9c2-82f8-4aed-8b95-9ba38287a35a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:05.277244 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 391/778] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Nov 2 03:22:05 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:22:05.281483 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48688ce-19e1-4b8c-810b-6b2a31f68513 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:05.305518 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 388/779] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Nov 2 03:22:05 2022] POST /identity/v3/projects => generated 311 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 03:22:05.372342 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:05.408997 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48688ce-19e1-4b8c-810b-6b2a31f68513 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:05.445557 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddb0c91-0284-4cd1-83b9-1c155ed19c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:05.446184 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb0c91-0284-4cd1-83b9-1c155ed19c52 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:05.446520 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb0c91-0284-4cd1-83b9-1c155ed19c52 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:05.446792 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddb0c91-0284-4cd1-83b9-1c155ed19c52 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:05.447429 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddb0c91-0284-4cd1-83b9-1c155ed19c52 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:22:05.447727 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddb0c91-0284-4cd1-83b9-1c155ed19c52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:05.449161 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddb0c91-0284-4cd1-83b9-1c155ed19c52 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:05.454749 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 392/780] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Nov 2 03:22:05 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:22:05.460533 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b79d9c2-82f8-4aed-8b95-9ba38287a35a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:05.570686 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b79d9c2-82f8-4aed-8b95-9ba38287a35a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:05.604863 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d3c4fa-4059-456c-ae2f-b71dc7b03a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:05.605479 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3c4fa-4059-456c-ae2f-b71dc7b03a8d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:05.605754 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3c4fa-4059-456c-ae2f-b71dc7b03a8d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:05.606021 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d3c4fa-4059-456c-ae2f-b71dc7b03a8d None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:05.606642 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d3c4fa-4059-456c-ae2f-b71dc7b03a8d None admin] RBAC: Adding query filter params (type=compute) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:22:05.606938 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d3c4fa-4059-456c-ae2f-b71dc7b03a8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:05.608305 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d3c4fa-4059-456c-ae2f-b71dc7b03a8d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:05.618500 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 389/781] 10.222.0.26 () {56 vars in 1112 bytes} [Wed Nov 2 03:22:05 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:22:05.619991 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddb0c91-0284-4cd1-83b9-1c155ed19c52 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:05.723305 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddb0c91-0284-4cd1-83b9-1c155ed19c52 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:05.758833 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca24cb2-f9ff-47ed-a4b0-293dbee8536f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:05.759443 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca24cb2-f9ff-47ed-a4b0-293dbee8536f None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:05.759718 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca24cb2-f9ff-47ed-a4b0-293dbee8536f None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:05.760005 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca24cb2-f9ff-47ed-a4b0-293dbee8536f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:05.760679 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca24cb2-f9ff-47ed-a4b0-293dbee8536f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:05.762021 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca24cb2-f9ff-47ed-a4b0-293dbee8536f None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:05.766574 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:05.783944 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 393/782] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:22:05 2022] POST /identity/v3/endpoints => generated 334 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:22:06.213675 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc5baa-fd1e-4ba1-9771-77ff767990f1 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:06.214204 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc5baa-fd1e-4ba1-9771-77ff767990f1 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:06.214204 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dc5baa-fd1e-4ba1-9771-77ff767990f1 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:06.215752 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 390/783] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:22:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:22:06.221699 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c5e548-ce90-4ec4-a8be-4bea78f57189 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:06.221992 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c5e548-ce90-4ec4-a8be-4bea78f57189 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:06.222269 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c5e548-ce90-4ec4-a8be-4bea78f57189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:06.241428 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-d9c5e548-ce90-4ec4-a8be-4bea78f57189 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:22:06.242660 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d9c5e548-ce90-4ec4-a8be-4bea78f57189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:22:06.319864 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 394/784] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:22:06 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 03:22:06.590429 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dc5baa-fd1e-4ba1-9771-77ff767990f1 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:06.636479 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:06.697519 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dc5baa-fd1e-4ba1-9771-77ff767990f1 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:06.732959 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b5edd1-7a56-466d-b077-20f7b95a1af0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:06.733606 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b5edd1-7a56-466d-b077-20f7b95a1af0 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:06.733885 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b5edd1-7a56-466d-b077-20f7b95a1af0 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:06.734157 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b5edd1-7a56-466d-b077-20f7b95a1af0 None admin] PATH_INFO: `/v3/services/image` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:06.734741 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b5edd1-7a56-466d-b077-20f7b95a1af0 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:06.736152 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b5edd1-7a56-466d-b077-20f7b95a1af0 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:06.745554 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-83b5edd1-7a56-466d-b077-20f7b95a1af0 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 03:22:06.745554 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:22:06.745554 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 03:22:06.745554 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 03:22:06.745554 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 03:22:06.745554 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 03:22:06.745554 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 03:22:06.745554 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 03:22:06.745554 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 03:22:06.745554 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:22:06.745554 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 03:22:06.745554 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:22:06.745554 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:22:06.745554 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:22:06.745554 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:22:06.745554 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:22:06.745554 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:22:06.746560 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:22:06.746560 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:22:06.746560 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:22:06.746560 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:22:06.746560 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:22:06.746560 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:22:06.746560 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 03:22:06.746560 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 03:22:06.746560 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 03:22:06.746560 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 03:22:06.746560 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:22:06.746560 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:22:06.746560 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:22:06.746560 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:22:06.746560 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:22:06.746560 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:22:06.749056 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:22:06.749056 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:22:06.749056 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:22:06.749056 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:22:06.749056 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:22:06.749056 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:22:06.749056 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:22:06.749056 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:22:06.749056 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:22:06.749056 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:22:06.749056 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 03:22:06.749056 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 03:22:06.749056 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Nov 02 03:22:06.749056 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:22:06.749056 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 391/785] 10.222.0.26 () {56 vars in 1105 bytes} [Wed Nov 2 03:22:06 2022] GET /identity/v3/services/image => generated 86 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:22:06.752919 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c5e548-ce90-4ec4-a8be-4bea78f57189 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:06.850895 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c5e548-ce90-4ec4-a8be-4bea78f57189 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:06.867571 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:06.886901 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24361ef-ccd2-4947-8db4-dd233d0d47f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:06.887508 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24361ef-ccd2-4947-8db4-dd233d0d47f1 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:06.887787 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24361ef-ccd2-4947-8db4-dd233d0d47f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:06.888095 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24361ef-ccd2-4947-8db4-dd233d0d47f1 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:06.888721 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24361ef-ccd2-4947-8db4-dd233d0d47f1 None admin] RBAC: Adding query filter params (name=image) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:22:06.889019 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24361ef-ccd2-4947-8db4-dd233d0d47f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:06.890366 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24361ef-ccd2-4947-8db4-dd233d0d47f1 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:06.899716 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 395/786] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:22:06 2022] GET /identity/v3/services?name=image => generated 122 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:22:06.901243 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b5edd1-7a56-466d-b077-20f7b95a1af0 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:07.002427 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b5edd1-7a56-466d-b077-20f7b95a1af0 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:07.004800 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:07.037834 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929eb845-be36-4e63-9b81-9305c08c74ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:07.038447 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929eb845-be36-4e63-9b81-9305c08c74ec None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:07.038740 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929eb845-be36-4e63-9b81-9305c08c74ec None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:07.038969 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929eb845-be36-4e63-9b81-9305c08c74ec None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:07.039589 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929eb845-be36-4e63-9b81-9305c08c74ec None admin] RBAC: Adding query filter params (type=image) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:22:07.039887 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929eb845-be36-4e63-9b81-9305c08c74ec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:07.041303 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929eb845-be36-4e63-9b81-9305c08c74ec None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:07.046312 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 392/787] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:22:06 2022] GET /identity/v3/services?type=image => generated 350 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:22:07.052145 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24361ef-ccd2-4947-8db4-dd233d0d47f1 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:07.154188 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24361ef-ccd2-4947-8db4-dd233d0d47f1 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:07.188432 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fe5d5a6-12ba-45b7-8d06-c25aee14b7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:07.189034 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5d5a6-12ba-45b7-8d06-c25aee14b7f1 None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:07.189313 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5d5a6-12ba-45b7-8d06-c25aee14b7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:07.189584 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe5d5a6-12ba-45b7-8d06-c25aee14b7f1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:07.190256 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe5d5a6-12ba-45b7-8d06-c25aee14b7f1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:07.191611 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fe5d5a6-12ba-45b7-8d06-c25aee14b7f1 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:07.211995 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 396/788] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:22:07 2022] POST /identity/v3/endpoints => generated 327 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:22:07.526248 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2298b68b-a6af-4fac-a7f1-fab211bad7c4 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:07.526248 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2298b68b-a6af-4fac-a7f1-fab211bad7c4 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:07.526248 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2298b68b-a6af-4fac-a7f1-fab211bad7c4 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:07.526940 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 393/789] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:22:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:22:07.532343 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4cccaa-a72b-46bd-bb73-d94d3eef7610 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:07.532483 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4cccaa-a72b-46bd-bb73-d94d3eef7610 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:07.535348 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4cccaa-a72b-46bd-bb73-d94d3eef7610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:07.550936 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-2f4cccaa-a72b-46bd-bb73-d94d3eef7610 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:22:07.552163 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-2f4cccaa-a72b-46bd-bb73-d94d3eef7610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:22:07.581721 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:07.628781 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 397/790] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:22:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 03:22:07.896854 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2298b68b-a6af-4fac-a7f1-fab211bad7c4 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:07.998067 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e350c-4f4c-4358-b3cf-3b725362b6e8 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:07.998694 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e350c-4f4c-4358-b3cf-3b725362b6e8 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:07.998793 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690e350c-4f4c-4358-b3cf-3b725362b6e8 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:08.000546 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 398/791] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:22:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:22:08.005383 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49ce4f-1a43-45a4-9117-a1b42902bb7f None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:08.006278 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49ce4f-1a43-45a4-9117-a1b42902bb7f None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:08.006555 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49ce4f-1a43-45a4-9117-a1b42902bb7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:08.008852 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2298b68b-a6af-4fac-a7f1-fab211bad7c4 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:08.024549 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-7a49ce4f-1a43-45a4-9117-a1b42902bb7f None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:22:08.025716 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-7a49ce4f-1a43-45a4-9117-a1b42902bb7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:22:08.044473 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19627c01-dd02-4792-93e4-6baa8c3d3196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:08.045104 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19627c01-dd02-4792-93e4-6baa8c3d3196 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:08.045387 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19627c01-dd02-4792-93e4-6baa8c3d3196 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:08.045653 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19627c01-dd02-4792-93e4-6baa8c3d3196 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:08.046765 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19627c01-dd02-4792-93e4-6baa8c3d3196 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:08.048221 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19627c01-dd02-4792-93e4-6baa8c3d3196 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:08.049792 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 394/792] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:22:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:22:08.055603 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19627c01-dd02-4792-93e4-6baa8c3d3196 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:08.093576 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:08.097986 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 399/793] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:22:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 03:22:08.161993 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19627c01-dd02-4792-93e4-6baa8c3d3196 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:08.194956 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f656f52b-ffa3-4c83-a1de-3149932c775c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:08.195557 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f656f52b-ffa3-4c83-a1de-3149932c775c None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:08.195833 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f656f52b-ffa3-4c83-a1de-3149932c775c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:08.196129 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f656f52b-ffa3-4c83-a1de-3149932c775c None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:08.196767 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f656f52b-ffa3-4c83-a1de-3149932c775c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:08.198158 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f656f52b-ffa3-4c83-a1de-3149932c775c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:08.227981 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 395/794] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Nov 2 03:22:08 2022] POST /identity/v3/users => generated 285 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 03:22:08.366803 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a49ce4f-1a43-45a4-9117-a1b42902bb7f None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:08.463284 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:08.473361 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a49ce4f-1a43-45a4-9117-a1b42902bb7f None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:08.509884 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e532712-6ad6-4532-a353-c4c33db90c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:08.510563 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e532712-6ad6-4532-a353-c4c33db90c23 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:08.510834 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e532712-6ad6-4532-a353-c4c33db90c23 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:08.511107 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e532712-6ad6-4532-a353-c4c33db90c23 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:08.521574 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e532712-6ad6-4532-a353-c4c33db90c23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:22:08.526532 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-6e532712-6ad6-4532-a353-c4c33db90c23 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 03:22:08.526532 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:22:08.526532 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:22:08.526532 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:22:08.526532 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:22:08.526532 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:22:08.526532 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:22:08.526532 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:22:08.526532 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:22:08.526532 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:22:08.526532 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:22:08.526532 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:22:08.526532 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:22:08.526532 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:22:08.526532 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:22:08.526532 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:22:08.527527 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:22:08.527527 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:22:08.527527 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:22:08.527527 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:22:08.527527 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:22:08.527527 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:22:08.527527 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:22:08.527527 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:22:08.527527 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:22:08.527527 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:22:08.527527 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:22:08.527527 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:22:08.527527 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:22:08.527527 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:22:08.527527 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:22:08.527527 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:22:08.528456 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:22:08.528456 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:22:08.528456 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:22:08.528456 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:22:08.528456 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:22:08.528456 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:22:08.528456 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:22:08.528456 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:22:08.528456 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:22:08.528456 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:22:08.528456 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 03:22:08.528456 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:22:08.528456 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 400/795] 10.222.0.26 () {56 vars in 1120 bytes} [Wed Nov 2 03:22:08 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 03:22:08.533645 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f656f52b-ffa3-4c83-a1de-3149932c775c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:08.628732 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:08.640130 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f656f52b-ffa3-4c83-a1de-3149932c775c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:08.673376 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e38463-0f3a-452a-85be-5e8ac961e37f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:08.673984 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e38463-0f3a-452a-85be-5e8ac961e37f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:08.674263 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e38463-0f3a-452a-85be-5e8ac961e37f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:08.674535 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e38463-0f3a-452a-85be-5e8ac961e37f None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:08.675191 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e38463-0f3a-452a-85be-5e8ac961e37f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:22:08.675493 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e38463-0f3a-452a-85be-5e8ac961e37f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:08.676881 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e38463-0f3a-452a-85be-5e8ac961e37f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:08.686136 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 396/796] 10.222.0.26 () {56 vars in 1115 bytes} [Wed Nov 2 03:22:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:22:08.691635 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e532712-6ad6-4532-a353-c4c33db90c23 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:08.796591 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e532712-6ad6-4532-a353-c4c33db90c23 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:08.830634 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3967f206-4fec-4f55-9210-eaee51b2846e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:08.831264 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3967f206-4fec-4f55-9210-eaee51b2846e None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:08.831543 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3967f206-4fec-4f55-9210-eaee51b2846e None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:08.831813 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3967f206-4fec-4f55-9210-eaee51b2846e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:08.841851 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3967f206-4fec-4f55-9210-eaee51b2846e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:08.843292 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3967f206-4fec-4f55-9210-eaee51b2846e None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:08.858208 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-3967f206-4fec-4f55-9210-eaee51b2846e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:22:08.858208 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:22:08.858208 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:22:08.858208 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:22:08.858208 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:22:08.858208 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:22:08.858208 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:22:08.858208 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:22:08.858208 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:22:08.858208 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:22:08.858208 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:22:08.858208 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:22:08.858208 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 03:22:08.858208 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 03:22:08.858208 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 03:22:08.858208 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 03:22:08.859180 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:22:08.859180 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:22:08.859180 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:22:08.859180 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:22:08.859180 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:22:08.859180 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:22:08.859180 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:22:08.859180 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:22:08.859180 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:22:08.859180 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:22:08.859180 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:22:08.859180 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:22:08.859180 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:22:08.859180 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:22:08.859180 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:22:08.859180 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:22:08.859180 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 03:22:08.865754 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 03:22:08.865754 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:22:08.865754 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:22:08.865754 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 401/797] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:22:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 03:22:08.867420 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e38463-0f3a-452a-85be-5e8ac961e37f None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:08.965808 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e38463-0f3a-452a-85be-5e8ac961e37f None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:09.000337 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d1284a-d14f-43dc-b96f-233379a62c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:09.000954 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d1284a-d14f-43dc-b96f-233379a62c03 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:09.001231 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d1284a-d14f-43dc-b96f-233379a62c03 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:09.001504 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d1284a-d14f-43dc-b96f-233379a62c03 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:09.002137 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d1284a-d14f-43dc-b96f-233379a62c03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:22:09.002437 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d1284a-d14f-43dc-b96f-233379a62c03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:09.003790 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d1284a-d14f-43dc-b96f-233379a62c03 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:09.020015 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 397/798] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Nov 2 03:22:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:22:09.022601 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3967f206-4fec-4f55-9210-eaee51b2846e None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:09.026550 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:09.125553 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3967f206-4fec-4f55-9210-eaee51b2846e None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:09.159417 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58d622a-16a5-4c91-82cb-08f1589e8806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:09.160096 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d622a-16a5-4c91-82cb-08f1589e8806 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:09.160377 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d622a-16a5-4c91-82cb-08f1589e8806 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:09.160646 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58d622a-16a5-4c91-82cb-08f1589e8806 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:09.169694 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58d622a-16a5-4c91-82cb-08f1589e8806 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:09.171146 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58d622a-16a5-4c91-82cb-08f1589e8806 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:09.182800 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-a58d622a-16a5-4c91-82cb-08f1589e8806 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Nov 02 03:22:09.182800 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:22:09.182800 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:22:09.182800 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:22:09.182800 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:22:09.182800 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:22:09.182800 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:22:09.182800 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:22:09.182800 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:22:09.182800 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:22:09.182800 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:22:09.182800 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:22:09.182800 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 03:22:09.182800 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 03:22:09.182800 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 03:22:09.182800 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 03:22:09.184033 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:22:09.184033 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:22:09.184033 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 03:22:09.184033 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 03:22:09.184033 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 03:22:09.184033 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 03:22:09.184033 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:22:09.184033 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:22:09.184033 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:22:09.184033 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:22:09.184033 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:22:09.184033 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:22:09.184033 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:22:09.184033 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:22:09.184033 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:22:09.184033 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:22:09.184033 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:22:09.184033 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:22:09.185465 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:22:09.185465 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:22:09.185465 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 03:22:09.185465 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 03:22:09.185465 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 03:22:09.185465 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 03:22:09.185465 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 03:22:09.185465 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 03:22:09.185465 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Nov 02 03:22:09.185465 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:22:09.188107 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 402/799] 10.222.0.26 () {56 vars in 1093 bytes} [Wed Nov 2 03:22:09 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:22:09.189644 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d1284a-d14f-43dc-b96f-233379a62c03 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:09.206869 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:09.298595 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d1284a-d14f-43dc-b96f-233379a62c03 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:09.331923 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acce7048-ed90-413f-9352-c49e4383e3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:09.332565 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce7048-ed90-413f-9352-c49e4383e3cc None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:09.332847 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce7048-ed90-413f-9352-c49e4383e3cc None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:09.333123 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acce7048-ed90-413f-9352-c49e4383e3cc None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:09.334372 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acce7048-ed90-413f-9352-c49e4383e3cc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:22:09.334681 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acce7048-ed90-413f-9352-c49e4383e3cc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:09.336264 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acce7048-ed90-413f-9352-c49e4383e3cc None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:09.358561 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 398/800] 10.222.0.26 () {56 vars in 1133 bytes} [Wed Nov 2 03:22:09 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:22:09.359753 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58d622a-16a5-4c91-82cb-08f1589e8806 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:09.381535 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8da060-4e6b-40e7-bba2-b51bae3daeb1 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:09.381743 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8da060-4e6b-40e7-bba2-b51bae3daeb1 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:09.382017 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8da060-4e6b-40e7-bba2-b51bae3daeb1 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:09.383473 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 399/801] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:22:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 03:22:09.388293 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc0127-2410-4c93-9f2c-0725bf85f270 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:09.388575 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc0127-2410-4c93-9f2c-0725bf85f270 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:09.388848 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc0127-2410-4c93-9f2c-0725bf85f270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:09.406996 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-5bbc0127-2410-4c93-9f2c-0725bf85f270 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:22:09.408226 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5bbc0127-2410-4c93-9f2c-0725bf85f270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:22:09.427125 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:09.468784 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58d622a-16a5-4c91-82cb-08f1589e8806 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:09.474847 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 400/802] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:22:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 03:22:09.506646 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fb6760-d914-46ea-bdbb-7877245a2492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:09.507274 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb6760-d914-46ea-bdbb-7877245a2492 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:09.507550 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb6760-d914-46ea-bdbb-7877245a2492 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:09.507819 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb6760-d914-46ea-bdbb-7877245a2492 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:09.519269 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fb6760-d914-46ea-bdbb-7877245a2492 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:09.520731 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fb6760-d914-46ea-bdbb-7877245a2492 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:09.535381 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-98fb6760-d914-46ea-bdbb-7877245a2492 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:22:09.535381 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:22:09.535381 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:22:09.535381 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:22:09.535381 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:22:09.535381 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:22:09.535381 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:22:09.535381 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:22:09.535381 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:22:09.535381 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:22:09.535381 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:22:09.535381 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:22:09.535381 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 03:22:09.535381 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 03:22:09.535381 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 03:22:09.535381 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 03:22:09.536556 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:22:09.536556 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:22:09.536556 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:22:09.536556 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:22:09.536556 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:22:09.536556 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:22:09.536556 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:22:09.536556 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:22:09.536556 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:22:09.536556 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:22:09.536556 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:22:09.536556 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:22:09.536556 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:22:09.536556 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:22:09.536556 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:22:09.536556 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:22:09.536556 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 03:22:09.538293 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 03:22:09.538293 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:22:09.538293 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:22:09.538293 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 403/803] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:22:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 03:22:09.542642 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc0127-2410-4c93-9f2c-0725bf85f270 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:09.545284 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:09.644007 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc0127-2410-4c93-9f2c-0725bf85f270 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:09.746496 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fb6760-d914-46ea-bdbb-7877245a2492 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:09.792585 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f03195f-7c3c-45f6-ba7e-e78e4289c995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:09.793194 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f03195f-7c3c-45f6-ba7e-e78e4289c995 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:09.793468 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f03195f-7c3c-45f6-ba7e-e78e4289c995 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:09.793737 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f03195f-7c3c-45f6-ba7e-e78e4289c995 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:09.794372 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f03195f-7c3c-45f6-ba7e-e78e4289c995 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:22:09.794671 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f03195f-7c3c-45f6-ba7e-e78e4289c995 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:09.796040 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f03195f-7c3c-45f6-ba7e-e78e4289c995 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:09.808000 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 401/804] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Nov 2 03:22:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:22:09.813202 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f03195f-7c3c-45f6-ba7e-e78e4289c995 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:09.864837 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fb6760-d914-46ea-bdbb-7877245a2492 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:09.878662 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:09.882453 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:09.904725 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a29b65d-b1e5-4329-bfff-cdcaaf6bb5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:09.905262 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a29b65d-b1e5-4329-bfff-cdcaaf6bb5df None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:09.905541 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a29b65d-b1e5-4329-bfff-cdcaaf6bb5df None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:09.906216 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a29b65d-b1e5-4329-bfff-cdcaaf6bb5df None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:09.906919 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a29b65d-b1e5-4329-bfff-cdcaaf6bb5df None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:09.908476 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a29b65d-b1e5-4329-bfff-cdcaaf6bb5df None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:09.917089 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.sql.core [None req-8a29b65d-b1e5-4329-bfff-cdcaaf6bb5df None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Nov 02 03:22:09.917089 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Nov 02 03:22:09.917089 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [parameters: {'id': 'c84af5f792494daab279bc0a4197705e', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Nov 02 03:22:09.917089 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81504) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Nov 02 03:22:09.918311 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f03195f-7c3c-45f6-ba7e-e78e4289c995 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:09.924410 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-8a29b65d-b1e5-4329-bfff-cdcaaf6bb5df None admin] Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Nov 02 03:22:09.924410 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:22:09.924410 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 03:22:09.924410 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 03:22:09.924410 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 03:22:09.924410 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 03:22:09.924410 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 03:22:09.924410 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 03:22:09.924410 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 03:22:09.924410 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application conn.query(q) Nov 02 03:22:09.924410 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 03:22:09.924410 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 03:22:09.924410 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 03:22:09.924410 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application result.read() Nov 02 03:22:09.925292 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 03:22:09.925292 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 03:22:09.925292 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 03:22:09.925292 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 03:22:09.925292 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 03:22:09.925292 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 03:22:09.925292 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 03:22:09.925292 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 03:22:09.925292 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Nov 02 03:22:09.925292 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:22:09.925292 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Nov 02 03:22:09.925292 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:22:09.925292 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:22:09.925292 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Nov 02 03:22:09.925292 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return method(*args, **kwargs) Nov 02 03:22:09.925292 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Nov 02 03:22:09.925292 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return project_ref.to_dict() Nov 02 03:22:09.925292 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 03:22:09.927290 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application next(self.gen) Nov 02 03:22:09.927290 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1068, in _transaction_scope Nov 02 03:22:09.927290 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application yield resource Nov 02 03:22:09.927290 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Nov 02 03:22:09.927290 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application next(self.gen) Nov 02 03:22:09.927290 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 670, in _session Nov 02 03:22:09.927290 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self.session.rollback() Nov 02 03:22:09.927290 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Nov 02 03:22:09.927290 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self.force_reraise() Nov 02 03:22:09.927290 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Nov 02 03:22:09.927290 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise self.value Nov 02 03:22:09.927290 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 667, in _session Nov 02 03:22:09.927290 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Nov 02 03:22:09.927290 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 695, in _end_session_transaction Nov 02 03:22:09.927290 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application session.commit() Nov 02 03:22:09.927290 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Nov 02 03:22:09.927290 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Nov 02 03:22:09.928970 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Nov 02 03:22:09.928970 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._prepare_impl() Nov 02 03:22:09.928970 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Nov 02 03:22:09.928970 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self.session.flush() Nov 02 03:22:09.928970 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Nov 02 03:22:09.928970 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._flush(objects) Nov 02 03:22:09.928970 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Nov 02 03:22:09.928970 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Nov 02 03:22:09.928970 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Nov 02 03:22:09.928970 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application compat.raise_( Nov 02 03:22:09.928970 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 03:22:09.928970 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception Nov 02 03:22:09.928970 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Nov 02 03:22:09.928970 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application flush_context.execute() Nov 02 03:22:09.928970 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Nov 02 03:22:09.928970 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rec.execute(self) Nov 02 03:22:09.928970 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Nov 02 03:22:09.929991 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Nov 02 03:22:09.929991 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Nov 02 03:22:09.929991 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application _emit_insert_statements( Nov 02 03:22:09.929991 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Nov 02 03:22:09.929991 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application c = connection._execute_20( Nov 02 03:22:09.929991 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Nov 02 03:22:09.929991 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Nov 02 03:22:09.929991 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Nov 02 03:22:09.929991 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Nov 02 03:22:09.929991 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Nov 02 03:22:09.929991 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ret = self._execute_context( Nov 02 03:22:09.929991 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Nov 02 03:22:09.929991 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Nov 02 03:22:09.929991 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Nov 02 03:22:09.929991 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Nov 02 03:22:09.931740 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Nov 02 03:22:09.931740 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception Nov 02 03:22:09.931740 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Nov 02 03:22:09.931740 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self.dialect.do_execute( Nov 02 03:22:09.931740 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Nov 02 03:22:09.931740 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Nov 02 03:22:09.931740 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Nov 02 03:22:09.931740 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application result = self._query(query) Nov 02 03:22:09.931740 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Nov 02 03:22:09.931740 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application conn.query(q) Nov 02 03:22:09.931740 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Nov 02 03:22:09.931740 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Nov 02 03:22:09.931740 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Nov 02 03:22:09.931740 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application result.read() Nov 02 03:22:09.931740 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Nov 02 03:22:09.931740 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Nov 02 03:22:09.931740 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Nov 02 03:22:09.933042 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application packet.raise_for_error() Nov 02 03:22:09.933042 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Nov 02 03:22:09.933042 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Nov 02 03:22:09.933042 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Nov 02 03:22:09.933042 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Nov 02 03:22:09.933042 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry '<>-Default' for key 'project.ixu_project_name_domain_id'") Nov 02 03:22:09.933042 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Nov 02 03:22:09.933042 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [parameters: {'id': 'c84af5f792494daab279bc0a4197705e', 'name': 'Default', 'domain_id': '<>', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': None, 'is_domain': 1}] Nov 02 03:22:09.933042 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Nov 02 03:22:09.933042 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:22:09.933042 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 03:22:09.933042 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:22:09.933042 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:22:09.933042 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Nov 02 03:22:09.933042 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Nov 02 03:22:09.934046 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Nov 02 03:22:09.934046 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Nov 02 03:22:09.934046 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name Default. Nov 02 03:22:09.934046 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:22:09.934046 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 03:22:09.934046 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:22:09.934046 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:22:09.934046 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:22:09.934046 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:22:09.934046 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:22:09.934046 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:22:09.934046 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:22:09.934046 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:22:09.934046 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:22:09.934046 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:22:09.934046 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:22:09.934046 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:22:09.936251 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 128, in post Nov 02 03:22:09.936251 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_domain( Nov 02 03:22:09.936251 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:22:09.936251 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:22:09.936251 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 765, in create_domain Nov 02 03:22:09.936251 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application is_domain_project = self.create_project( Nov 02 03:22:09.936251 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:22:09.936251 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:22:09.936251 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Nov 02 03:22:09.936251 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.Conflict( Nov 02 03:22:09.936251 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects acting as domains with the same name: Default. Nov 02 03:22:09.936251 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:22:09.936251 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 404/805] 10.222.0.26 () {60 vars in 1137 bytes} [Wed Nov 2 03:22:09 2022] POST /identity/v3/domains => generated 190 bytes in 180 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Nov 02 03:22:09.936251 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a29b65d-b1e5-4329-bfff-cdcaaf6bb5df None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:09.962773 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7a90c3-dca0-4d6d-af4c-fcaf9d881325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:09.963719 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7a90c3-dca0-4d6d-af4c-fcaf9d881325 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:09.964018 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7a90c3-dca0-4d6d-af4c-fcaf9d881325 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:09.964447 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7a90c3-dca0-4d6d-af4c-fcaf9d881325 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:09.971518 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7a90c3-dca0-4d6d-af4c-fcaf9d881325 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:09.972977 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7a90c3-dca0-4d6d-af4c-fcaf9d881325 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:09.984179 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-ae7a90c3-dca0-4d6d-af4c-fcaf9d881325 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 03:22:09.984179 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:22:09.984179 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:22:09.984179 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:22:09.984179 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:22:09.984179 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:22:09.984179 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:22:09.984179 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:22:09.984179 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:22:09.984179 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:22:09.984179 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:22:09.984179 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:22:09.984179 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 03:22:09.984179 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 03:22:09.984179 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 03:22:09.985086 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 03:22:09.985086 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:22:09.985086 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:22:09.985086 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:22:09.985086 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:22:09.985086 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:22:09.985086 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:22:09.985086 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:22:09.985086 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:22:09.985086 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:22:09.985086 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:22:09.985086 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:22:09.985086 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:22:09.985086 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:22:09.985086 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:22:09.985086 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:22:09.985086 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:22:09.985975 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 03:22:09.985975 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 03:22:09.985975 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 03:22:09.985975 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 03:22:09.985975 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 03:22:09.985975 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 03:22:09.985975 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 03:22:09.985975 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:22:09.985975 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 402/806] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Nov 2 03:22:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 03:22:09.992158 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7a90c3-dca0-4d6d-af4c-fcaf9d881325 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:10.039941 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a29b65d-b1e5-4329-bfff-cdcaaf6bb5df None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:10.075547 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ac49f1-36bf-4cdb-a6da-a43f6164c4da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:10.076525 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac49f1-36bf-4cdb-a6da-a43f6164c4da None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:10.076943 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac49f1-36bf-4cdb-a6da-a43f6164c4da None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:10.077451 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ac49f1-36bf-4cdb-a6da-a43f6164c4da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:10.089382 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ac49f1-36bf-4cdb-a6da-a43f6164c4da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:10.091050 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ac49f1-36bf-4cdb-a6da-a43f6164c4da None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:10.098574 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7a90c3-dca0-4d6d-af4c-fcaf9d881325 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:10.106070 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-f6ac49f1-36bf-4cdb-a6da-a43f6164c4da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:22:10.106070 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:22:10.106070 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:22:10.106070 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:22:10.106070 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:22:10.106070 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:22:10.106070 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:22:10.106070 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:22:10.106070 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:22:10.106070 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:22:10.106070 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:22:10.106070 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:22:10.106070 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 03:22:10.106070 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 03:22:10.106070 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 03:22:10.106070 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 03:22:10.107035 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:22:10.107035 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:22:10.107035 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:22:10.107035 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:22:10.107035 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:22:10.107035 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:22:10.107035 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:22:10.107035 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:22:10.107035 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:22:10.107035 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:22:10.107035 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:22:10.107035 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:22:10.107035 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:22:10.107035 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:22:10.107035 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:22:10.107035 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:22:10.107035 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 03:22:10.107929 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 03:22:10.107929 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:22:10.107929 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:22:10.107929 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 405/807] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:22:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 03:22:10.113708 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ac49f1-36bf-4cdb-a6da-a43f6164c4da None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:10.125690 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:10.137762 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286bbbc7-915c-47be-b6ae-54f19a260247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:10.138369 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286bbbc7-915c-47be-b6ae-54f19a260247 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:10.138643 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286bbbc7-915c-47be-b6ae-54f19a260247 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:10.138911 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286bbbc7-915c-47be-b6ae-54f19a260247 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:10.139623 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286bbbc7-915c-47be-b6ae-54f19a260247 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:22:10.139923 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286bbbc7-915c-47be-b6ae-54f19a260247 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:10.141427 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286bbbc7-915c-47be-b6ae-54f19a260247 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:10.149751 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:10.158881 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 403/808] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Nov 2 03:22:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:22:10.159884 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286bbbc7-915c-47be-b6ae-54f19a260247 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:10.231434 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ac49f1-36bf-4cdb-a6da-a43f6164c4da None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:10.266488 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286bbbc7-915c-47be-b6ae-54f19a260247 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:10.272087 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8d95b8-0aa4-44b3-8c75-89fb2a7c9b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:10.272704 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d95b8-0aa4-44b3-8c75-89fb2a7c9b81 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:10.272986 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d95b8-0aa4-44b3-8c75-89fb2a7c9b81 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:10.273260 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8d95b8-0aa4-44b3-8c75-89fb2a7c9b81 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:10.274319 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8d95b8-0aa4-44b3-8c75-89fb2a7c9b81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:22:10.274630 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8d95b8-0aa4-44b3-8c75-89fb2a7c9b81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:10.276409 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8d95b8-0aa4-44b3-8c75-89fb2a7c9b81 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:10.292327 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 406/809] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Nov 2 03:22:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:22:10.305172 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df41ea67-251a-44e1-b58c-1ede3694ceb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:10.306178 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df41ea67-251a-44e1-b58c-1ede3694ceb1 None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:10.306637 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df41ea67-251a-44e1-b58c-1ede3694ceb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:10.307117 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df41ea67-251a-44e1-b58c-1ede3694ceb1 None admin] PATH_INFO: `/v3/projects/d0b5e7017ac64ae6ab62c1a05aa54b72/users/44961b06151c4da382fcd54dd1f1e146/roles/c92260a73f5748618d4d4cbae46e5777` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:10.309649 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df41ea67-251a-44e1-b58c-1ede3694ceb1 None admin] RBAC: Authorizing `identity:create_grant(project_id=d0b5e7017ac64ae6ab62c1a05aa54b72, user_id=44961b06151c4da382fcd54dd1f1e146, role_id=c92260a73f5748618d4d4cbae46e5777)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:10.311391 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df41ea67-251a-44e1-b58c-1ede3694ceb1 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:10.328921 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 404/810] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Nov 2 03:22:10 2022] PUT /identity/v3/projects/d0b5e7017ac64ae6ab62c1a05aa54b72/users/44961b06151c4da382fcd54dd1f1e146/roles/c92260a73f5748618d4d4cbae46e5777 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 03:22:10.449459 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88174719-3a21-4abe-8dcb-e32fb8e8ac61 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:10.449661 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88174719-3a21-4abe-8dcb-e32fb8e8ac61 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:10.449925 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88174719-3a21-4abe-8dcb-e32fb8e8ac61 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:10.451469 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 407/811] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:22:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:22:10.457271 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1e2447-e5f8-4d47-a88f-226794aa3b86 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:10.457580 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1e2447-e5f8-4d47-a88f-226794aa3b86 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:10.457820 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1e2447-e5f8-4d47-a88f-226794aa3b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:10.477648 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-8f1e2447-e5f8-4d47-a88f-226794aa3b86 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:22:10.478860 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8f1e2447-e5f8-4d47-a88f-226794aa3b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:22:10.519662 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:10.555986 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 405/812] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:22:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 03:22:10.823043 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88174719-3a21-4abe-8dcb-e32fb8e8ac61 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:10.914213 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:10.924294 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88174719-3a21-4abe-8dcb-e32fb8e8ac61 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:10.957455 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be73c13-054e-427b-b26c-2fc2425aa652 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:10.958081 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be73c13-054e-427b-b26c-2fc2425aa652 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:10.958359 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be73c13-054e-427b-b26c-2fc2425aa652 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:10.958630 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be73c13-054e-427b-b26c-2fc2425aa652 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:10.970005 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be73c13-054e-427b-b26c-2fc2425aa652 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:22:10.974780 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-7be73c13-054e-427b-b26c-2fc2425aa652 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 03:22:10.974780 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:22:10.974780 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:22:10.974780 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:22:10.974780 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:22:10.974780 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:22:10.974780 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:22:10.974780 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:22:10.974780 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:22:10.974780 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:22:10.974780 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:22:10.974780 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:22:10.974780 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:22:10.974780 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:22:10.974780 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:22:10.974780 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:22:10.975901 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:22:10.975901 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:22:10.975901 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:22:10.975901 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:22:10.975901 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:22:10.975901 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:22:10.975901 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:22:10.975901 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:22:10.975901 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:22:10.975901 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:22:10.975901 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:22:10.975901 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:22:10.975901 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:22:10.975901 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:22:10.975901 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:22:10.975901 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:22:10.976925 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:22:10.976925 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:22:10.976925 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:22:10.976925 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:22:10.976925 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:22:10.976925 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:22:10.976925 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:22:10.976925 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:22:10.976925 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:22:10.976925 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:22:10.976925 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 03:22:10.976925 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:22:10.976925 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 408/813] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Nov 2 03:22:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:22:10.981783 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1e2447-e5f8-4d47-a88f-226794aa3b86 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:11.077986 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1e2447-e5f8-4d47-a88f-226794aa3b86 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:11.110295 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59931abe-2fea-482c-a0cc-a964d949ad24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:11.110893 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59931abe-2fea-482c-a0cc-a964d949ad24 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:11.111164 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59931abe-2fea-482c-a0cc-a964d949ad24 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:11.111427 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59931abe-2fea-482c-a0cc-a964d949ad24 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:11.112110 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59931abe-2fea-482c-a0cc-a964d949ad24 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:22:11.112653 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59931abe-2fea-482c-a0cc-a964d949ad24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:11.114000 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59931abe-2fea-482c-a0cc-a964d949ad24 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:11.122007 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 406/814] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Nov 2 03:22:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:22:11.127783 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be73c13-054e-427b-b26c-2fc2425aa652 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:11.226896 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be73c13-054e-427b-b26c-2fc2425aa652 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:11.260446 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8606d3-3692-4d7a-a9bd-47d4b53ad188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:11.261070 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8606d3-3692-4d7a-a9bd-47d4b53ad188 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:11.261350 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8606d3-3692-4d7a-a9bd-47d4b53ad188 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:11.261626 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8606d3-3692-4d7a-a9bd-47d4b53ad188 None admin] PATH_INFO: `/v3/users/459759940cc345dd9c9777e070288fc5` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:11.274340 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8606d3-3692-4d7a-a9bd-47d4b53ad188 None admin] RBAC: Authorizing `identity:get_user(user_id=459759940cc345dd9c9777e070288fc5)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:11.275792 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8606d3-3692-4d7a-a9bd-47d4b53ad188 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:11.277428 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 409/815] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Nov 2 03:22:11 2022] GET /identity/v3/users/459759940cc345dd9c9777e070288fc5 => generated 285 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:22:11.283157 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59931abe-2fea-482c-a0cc-a964d949ad24 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:11.317174 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:11.381185 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59931abe-2fea-482c-a0cc-a964d949ad24 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:11.413645 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159537d7-7ef6-4900-849a-609b99337f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:11.414263 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159537d7-7ef6-4900-849a-609b99337f6b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:11.414532 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159537d7-7ef6-4900-849a-609b99337f6b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:11.414794 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159537d7-7ef6-4900-849a-609b99337f6b None admin] PATH_INFO: `/v3/projects/8f338522ad2f4f2692365f36de706fc0` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:11.415878 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159537d7-7ef6-4900-849a-609b99337f6b None admin] RBAC: Authorizing `identity:get_project(project_id=8f338522ad2f4f2692365f36de706fc0)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:11.417304 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159537d7-7ef6-4900-849a-609b99337f6b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:11.418894 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 407/816] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Nov 2 03:22:11 2022] GET /identity/v3/projects/8f338522ad2f4f2692365f36de706fc0 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:22:11.424681 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8606d3-3692-4d7a-a9bd-47d4b53ad188 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:11.436621 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:11.521465 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8606d3-3692-4d7a-a9bd-47d4b53ad188 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:11.555506 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd85bec3-f913-40a6-932b-71d80b023307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:11.556199 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd85bec3-f913-40a6-932b-71d80b023307 None admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:11.556476 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd85bec3-f913-40a6-932b-71d80b023307 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:11.556748 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd85bec3-f913-40a6-932b-71d80b023307 None admin] PATH_INFO: `/v3/projects/8f338522ad2f4f2692365f36de706fc0/users/459759940cc345dd9c9777e070288fc5/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:11.558627 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd85bec3-f913-40a6-932b-71d80b023307 None admin] RBAC: Authorizing `identity:create_grant(project_id=8f338522ad2f4f2692365f36de706fc0, user_id=459759940cc345dd9c9777e070288fc5, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:11.560067 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd85bec3-f913-40a6-932b-71d80b023307 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:11.573356 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 410/817] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Nov 2 03:22:11 2022] PUT /identity/v3/projects/8f338522ad2f4f2692365f36de706fc0/users/459759940cc345dd9c9777e070288fc5/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:22:12.479781 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13fc952-d2af-41c4-a77b-772087d97fd8 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:12.480339 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13fc952-d2af-41c4-a77b-772087d97fd8 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:12.480429 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13fc952-d2af-41c4-a77b-772087d97fd8 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:12.481704 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 408/818] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:22:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:22:12.487112 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc944e27-9b5a-4b64-8c4f-0543bb00296e None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:12.487758 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc944e27-9b5a-4b64-8c4f-0543bb00296e None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:12.488059 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc944e27-9b5a-4b64-8c4f-0543bb00296e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:12.506770 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-dc944e27-9b5a-4b64-8c4f-0543bb00296e None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:22:12.507932 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-dc944e27-9b5a-4b64-8c4f-0543bb00296e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:22:12.571395 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:12.580821 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 411/819] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:22:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 03:22:12.843822 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13fc952-d2af-41c4-a77b-772087d97fd8 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:12.940962 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13fc952-d2af-41c4-a77b-772087d97fd8 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:12.942432 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:12.974227 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ee4e2-ef70-4236-bcc6-e459b4836a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:12.974842 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ee4e2-ef70-4236-bcc6-e459b4836a69 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:12.975120 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ee4e2-ef70-4236-bcc6-e459b4836a69 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:12.975391 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9ee4e2-ef70-4236-bcc6-e459b4836a69 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:12.985464 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9ee4e2-ef70-4236-bcc6-e459b4836a69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:22:12.990941 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-ec9ee4e2-ef70-4236-bcc6-e459b4836a69 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 03:22:12.990941 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:22:12.990941 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:22:12.990941 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:22:12.990941 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:22:12.990941 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:22:12.990941 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:22:12.990941 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:22:12.990941 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:22:12.990941 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:22:12.990941 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:22:12.990941 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:22:12.990941 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:22:12.990941 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:22:12.990941 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:22:12.990941 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:22:12.991996 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:22:12.991996 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:22:12.991996 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:22:12.991996 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:22:12.991996 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:22:12.991996 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:22:12.991996 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:22:12.991996 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:22:12.991996 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:22:12.991996 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:22:12.991996 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:22:12.991996 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:22:12.991996 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:22:12.991996 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:22:12.991996 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:22:12.991996 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:22:12.992865 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:22:12.992865 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:22:12.992865 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:22:12.992865 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:22:12.992865 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:22:12.992865 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:22:12.992865 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:22:12.992865 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:22:12.992865 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:22:12.992865 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:22:12.992865 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Nov 02 03:22:12.992865 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:22:12.992865 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 409/820] 10.222.0.26 () {56 vars in 1120 bytes} [Wed Nov 2 03:22:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:22:12.998086 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc944e27-9b5a-4b64-8c4f-0543bb00296e None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:13.092809 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc944e27-9b5a-4b64-8c4f-0543bb00296e None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:13.125887 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07d507e-7996-4413-b390-be3d60863de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:13.126435 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d507e-7996-4413-b390-be3d60863de0 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:13.126712 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d507e-7996-4413-b390-be3d60863de0 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:13.126981 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d507e-7996-4413-b390-be3d60863de0 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:13.127638 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07d507e-7996-4413-b390-be3d60863de0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:22:13.127937 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07d507e-7996-4413-b390-be3d60863de0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:13.129328 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07d507e-7996-4413-b390-be3d60863de0 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:13.137202 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 412/821] 10.222.0.26 () {56 vars in 1115 bytes} [Wed Nov 2 03:22:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:22:13.142993 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ee4e2-ef70-4236-bcc6-e459b4836a69 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:13.239622 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9ee4e2-ef70-4236-bcc6-e459b4836a69 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:13.273860 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9278657-0e4e-4318-bd9f-2e44f81a2fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:13.274490 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9278657-0e4e-4318-bd9f-2e44f81a2fcd None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:13.274766 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9278657-0e4e-4318-bd9f-2e44f81a2fcd None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:13.275034 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9278657-0e4e-4318-bd9f-2e44f81a2fcd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:13.285147 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9278657-0e4e-4318-bd9f-2e44f81a2fcd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:13.286583 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9278657-0e4e-4318-bd9f-2e44f81a2fcd None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:13.300810 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-c9278657-0e4e-4318-bd9f-2e44f81a2fcd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:22:13.300810 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:22:13.300810 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:22:13.300810 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:22:13.300810 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:22:13.300810 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:22:13.300810 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:22:13.300810 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:22:13.300810 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:22:13.300810 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:22:13.300810 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:22:13.300810 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:22:13.300810 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 03:22:13.300810 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 03:22:13.300810 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 03:22:13.300810 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 03:22:13.301910 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:22:13.301910 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:22:13.301910 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:22:13.301910 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:22:13.301910 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:22:13.301910 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:22:13.301910 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:22:13.301910 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:22:13.301910 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:22:13.301910 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:22:13.301910 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:22:13.301910 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:22:13.301910 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:22:13.301910 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:22:13.301910 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:22:13.301910 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:22:13.301910 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 03:22:13.303860 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 03:22:13.303860 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:22:13.303860 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:22:13.303860 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 410/822] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:22:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 03:22:13.308191 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07d507e-7996-4413-b390-be3d60863de0 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:13.368784 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:13.408388 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07d507e-7996-4413-b390-be3d60863de0 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:13.440975 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db287c-5c2e-4ced-810a-bedc3b5e9007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:13.441574 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db287c-5c2e-4ced-810a-bedc3b5e9007 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:13.441849 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db287c-5c2e-4ced-810a-bedc3b5e9007 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:13.442120 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00db287c-5c2e-4ced-810a-bedc3b5e9007 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:13.442751 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00db287c-5c2e-4ced-810a-bedc3b5e9007 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:22:13.443562 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00db287c-5c2e-4ced-810a-bedc3b5e9007 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:13.444981 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00db287c-5c2e-4ced-810a-bedc3b5e9007 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:13.457203 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 413/823] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Nov 2 03:22:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:22:13.463336 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9278657-0e4e-4318-bd9f-2e44f81a2fcd None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:13.478936 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:13.566188 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9278657-0e4e-4318-bd9f-2e44f81a2fcd None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:13.603293 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085326d0-303c-4830-94b0-11213714d3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:13.603929 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085326d0-303c-4830-94b0-11213714d3b9 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:13.604240 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085326d0-303c-4830-94b0-11213714d3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:13.604515 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085326d0-303c-4830-94b0-11213714d3b9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:13.613137 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085326d0-303c-4830-94b0-11213714d3b9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:13.614562 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085326d0-303c-4830-94b0-11213714d3b9 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:13.628586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-085326d0-303c-4830-94b0-11213714d3b9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Nov 02 03:22:13.628586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:22:13.628586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:22:13.628586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:22:13.628586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:22:13.628586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:22:13.628586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:22:13.628586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:22:13.628586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:22:13.628586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:22:13.628586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:22:13.628586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:22:13.628586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 03:22:13.628586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 03:22:13.628586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 03:22:13.628586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 03:22:13.629724 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:22:13.629724 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:22:13.629724 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 03:22:13.629724 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 03:22:13.629724 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 03:22:13.629724 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 03:22:13.629724 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:22:13.629724 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:22:13.629724 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:22:13.629724 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:22:13.629724 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:22:13.629724 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:22:13.629724 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:22:13.629724 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:22:13.629724 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:22:13.629724 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:22:13.629724 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:22:13.629724 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:22:13.630725 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:22:13.630725 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:22:13.630725 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 03:22:13.630725 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 03:22:13.630725 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 03:22:13.630725 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 03:22:13.630725 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 03:22:13.630725 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 03:22:13.630725 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Nov 02 03:22:13.630725 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:22:13.630725 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 411/824] 10.222.0.26 () {56 vars in 1093 bytes} [Wed Nov 2 03:22:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:22:13.635585 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db287c-5c2e-4ced-810a-bedc3b5e9007 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:13.684257 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e474e092-357e-48b9-9715-1a95748ecf2b None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:13.684436 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e474e092-357e-48b9-9715-1a95748ecf2b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:13.684753 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e474e092-357e-48b9-9715-1a95748ecf2b None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:13.686069 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 412/825] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:22:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 03:22:13.691034 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e984bd-4ce5-4b69-8451-eb15f8e82b65 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:13.691308 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e984bd-4ce5-4b69-8451-eb15f8e82b65 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:13.691575 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e984bd-4ce5-4b69-8451-eb15f8e82b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:13.709589 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-c1e984bd-4ce5-4b69-8451-eb15f8e82b65 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:22:13.710760 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c1e984bd-4ce5-4b69-8451-eb15f8e82b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:22:13.734235 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00db287c-5c2e-4ced-810a-bedc3b5e9007 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:13.757068 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:13.769356 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8642cf4c-2276-43ce-add2-5433c5077b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:13.769962 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8642cf4c-2276-43ce-add2-5433c5077b57 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:13.770239 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8642cf4c-2276-43ce-add2-5433c5077b57 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:13.770506 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8642cf4c-2276-43ce-add2-5433c5077b57 None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:13.771725 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8642cf4c-2276-43ce-add2-5433c5077b57 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:22:13.772059 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8642cf4c-2276-43ce-add2-5433c5077b57 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:13.773467 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8642cf4c-2276-43ce-add2-5433c5077b57 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:13.777266 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 413/826] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:22:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 03:22:13.794798 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 414/827] 10.222.0.26 () {56 vars in 1133 bytes} [Wed Nov 2 03:22:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:22:13.795863 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e984bd-4ce5-4b69-8451-eb15f8e82b65 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:13.799134 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:13.889753 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e984bd-4ce5-4b69-8451-eb15f8e82b65 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:13.922307 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd86e059-f917-4429-bc43-bf7fd8c627ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:13.922898 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd86e059-f917-4429-bc43-bf7fd8c627ad None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:13.923170 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd86e059-f917-4429-bc43-bf7fd8c627ad None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:13.923436 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd86e059-f917-4429-bc43-bf7fd8c627ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:13.986527 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd86e059-f917-4429-bc43-bf7fd8c627ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:13.988231 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd86e059-f917-4429-bc43-bf7fd8c627ad None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:13.990709 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:14.048736 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8642cf4c-2276-43ce-add2-5433c5077b57 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:14.086366 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 414/828] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Nov 2 03:22:13 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 03:22:14.092030 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd86e059-f917-4429-bc43-bf7fd8c627ad None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:14.148372 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8642cf4c-2276-43ce-add2-5433c5077b57 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:14.182809 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56858c91-1e02-4a56-b44f-edf3aebccef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:14.183431 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56858c91-1e02-4a56-b44f-edf3aebccef3 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:14.183703 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56858c91-1e02-4a56-b44f-edf3aebccef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:14.184001 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56858c91-1e02-4a56-b44f-edf3aebccef3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:14.188431 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd86e059-f917-4429-bc43-bf7fd8c627ad None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:14.190818 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:14.194248 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56858c91-1e02-4a56-b44f-edf3aebccef3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:22:14.199046 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-56858c91-1e02-4a56-b44f-edf3aebccef3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 03:22:14.199046 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:22:14.199046 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:22:14.199046 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:22:14.199046 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:22:14.199046 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:22:14.199046 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:22:14.199046 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:22:14.199046 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:22:14.199046 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:22:14.199046 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:22:14.199046 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:22:14.199046 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:22:14.199046 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:22:14.199046 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:22:14.199046 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:22:14.200043 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:22:14.200043 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:22:14.200043 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:22:14.200043 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:22:14.200043 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:22:14.200043 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:22:14.200043 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:22:14.200043 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:22:14.200043 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:22:14.200043 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:22:14.200043 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:22:14.200043 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:22:14.200043 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:22:14.200043 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:22:14.200043 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:22:14.200043 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:22:14.200914 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:22:14.200914 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:22:14.200914 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:22:14.200914 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:22:14.200914 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:22:14.200914 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:22:14.200914 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:22:14.200914 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:22:14.200914 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:22:14.200914 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:22:14.200914 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 03:22:14.200914 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:22:14.200914 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 415/829] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Nov 2 03:22:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 03:22:14.206128 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56858c91-1e02-4a56-b44f-edf3aebccef3 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:14.227473 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfd81e3-89e7-4f9e-b6fe-dcaeceb0a7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:14.228129 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfd81e3-89e7-4f9e-b6fe-dcaeceb0a7bc None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:14.228409 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfd81e3-89e7-4f9e-b6fe-dcaeceb0a7bc None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:14.228681 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfd81e3-89e7-4f9e-b6fe-dcaeceb0a7bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:14.238816 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfd81e3-89e7-4f9e-b6fe-dcaeceb0a7bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:14.240262 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfd81e3-89e7-4f9e-b6fe-dcaeceb0a7bc None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:14.247861 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:14.254566 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-9cfd81e3-89e7-4f9e-b6fe-dcaeceb0a7bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:22:14.254566 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:22:14.254566 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:22:14.254566 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:22:14.254566 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:22:14.254566 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:22:14.254566 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:22:14.254566 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:22:14.254566 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:22:14.254566 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:22:14.254566 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:22:14.254566 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:22:14.254566 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 03:22:14.254566 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 03:22:14.254566 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 03:22:14.254566 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 03:22:14.256057 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:22:14.256057 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:22:14.256057 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:22:14.256057 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:22:14.256057 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:22:14.256057 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:22:14.256057 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:22:14.256057 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:22:14.256057 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:22:14.256057 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:22:14.256057 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:22:14.256057 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:22:14.256057 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:22:14.256057 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:22:14.256057 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:22:14.256057 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:22:14.256057 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 03:22:14.257017 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 03:22:14.257017 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:22:14.257017 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:22:14.257017 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 415/830] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:22:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:22:14.261863 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfd81e3-89e7-4f9e-b6fe-dcaeceb0a7bc None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:14.310233 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56858c91-1e02-4a56-b44f-edf3aebccef3 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:14.343362 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22021162-ee94-410b-a3a6-66629e24c6c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:14.343974 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22021162-ee94-410b-a3a6-66629e24c6c8 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:14.344270 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22021162-ee94-410b-a3a6-66629e24c6c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:14.344543 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22021162-ee94-410b-a3a6-66629e24c6c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:14.345197 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22021162-ee94-410b-a3a6-66629e24c6c8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:22:14.345498 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22021162-ee94-410b-a3a6-66629e24c6c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:14.346859 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22021162-ee94-410b-a3a6-66629e24c6c8 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:14.355197 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 416/831] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Nov 2 03:22:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:22:14.361165 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22021162-ee94-410b-a3a6-66629e24c6c8 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:14.365205 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfd81e3-89e7-4f9e-b6fe-dcaeceb0a7bc None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:14.398395 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3f5dd8-09c2-4dbc-801e-44c00a3a20c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:14.399000 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f5dd8-09c2-4dbc-801e-44c00a3a20c1 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:14.399275 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f5dd8-09c2-4dbc-801e-44c00a3a20c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:14.399552 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f5dd8-09c2-4dbc-801e-44c00a3a20c1 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:14.400219 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3f5dd8-09c2-4dbc-801e-44c00a3a20c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:22:14.400521 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3f5dd8-09c2-4dbc-801e-44c00a3a20c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:14.401868 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3f5dd8-09c2-4dbc-801e-44c00a3a20c1 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:14.413266 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 416/832] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Nov 2 03:22:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:22:14.418744 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3f5dd8-09c2-4dbc-801e-44c00a3a20c1 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:14.436580 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:14.457741 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22021162-ee94-410b-a3a6-66629e24c6c8 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:14.459768 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:14.492433 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d543fde-fe49-4557-af11-b4b912514e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:14.493001 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d543fde-fe49-4557-af11-b4b912514e50 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:14.493284 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d543fde-fe49-4557-af11-b4b912514e50 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:14.493562 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d543fde-fe49-4557-af11-b4b912514e50 None admin] PATH_INFO: `/v3/users/459759940cc345dd9c9777e070288fc5` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:14.494728 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d543fde-fe49-4557-af11-b4b912514e50 None admin] RBAC: Authorizing `identity:get_user(user_id=459759940cc345dd9c9777e070288fc5)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:14.496203 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d543fde-fe49-4557-af11-b4b912514e50 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:14.497782 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 417/833] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Nov 2 03:22:14 2022] GET /identity/v3/users/459759940cc345dd9c9777e070288fc5 => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:22:14.503566 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d543fde-fe49-4557-af11-b4b912514e50 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:14.518664 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3f5dd8-09c2-4dbc-801e-44c00a3a20c1 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:14.553204 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c46b362-80cc-4097-a9df-7ad42bc2ae82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:14.553837 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c46b362-80cc-4097-a9df-7ad42bc2ae82 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:14.554114 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c46b362-80cc-4097-a9df-7ad42bc2ae82 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:14.554384 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c46b362-80cc-4097-a9df-7ad42bc2ae82 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:14.560870 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c46b362-80cc-4097-a9df-7ad42bc2ae82 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:14.562287 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c46b362-80cc-4097-a9df-7ad42bc2ae82 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:14.573521 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-7c46b362-80cc-4097-a9df-7ad42bc2ae82 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 03:22:14.573521 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:22:14.573521 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:22:14.573521 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:22:14.573521 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:22:14.573521 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:22:14.573521 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:22:14.573521 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:22:14.573521 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:22:14.573521 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:22:14.573521 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:22:14.573521 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:22:14.573521 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 03:22:14.573521 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 03:22:14.573521 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 03:22:14.574940 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 03:22:14.574940 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:22:14.574940 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:22:14.574940 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:22:14.574940 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:22:14.574940 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:22:14.574940 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:22:14.574940 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:22:14.574940 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:22:14.574940 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:22:14.574940 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:22:14.574940 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:22:14.574940 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:22:14.574940 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:22:14.574940 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:22:14.574940 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:22:14.574940 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:22:14.575847 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 03:22:14.575847 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 03:22:14.575847 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 03:22:14.575847 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 03:22:14.575847 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 03:22:14.575847 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 03:22:14.575847 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 03:22:14.575847 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:22:14.575847 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 417/834] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Nov 2 03:22:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 03:22:14.580398 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c46b362-80cc-4097-a9df-7ad42bc2ae82 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:14.603827 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d543fde-fe49-4557-af11-b4b912514e50 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:14.636855 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e126d7-7567-46fd-8689-e02a5c8a24dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:14.637447 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e126d7-7567-46fd-8689-e02a5c8a24dc None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:14.637722 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e126d7-7567-46fd-8689-e02a5c8a24dc None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:14.637994 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e126d7-7567-46fd-8689-e02a5c8a24dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:14.655534 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:14.676003 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:14.678357 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c46b362-80cc-4097-a9df-7ad42bc2ae82 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:14.706921 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e126d7-7567-46fd-8689-e02a5c8a24dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:14.708439 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e126d7-7567-46fd-8689-e02a5c8a24dc None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:14.711548 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf999639-3458-4895-b59c-c5600f1f4f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:14.712176 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf999639-3458-4895-b59c-c5600f1f4f94 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:14.712456 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf999639-3458-4895-b59c-c5600f1f4f94 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:14.712727 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf999639-3458-4895-b59c-c5600f1f4f94 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:14.713406 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf999639-3458-4895-b59c-c5600f1f4f94 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:22:14.713708 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf999639-3458-4895-b59c-c5600f1f4f94 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:14.715111 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf999639-3458-4895-b59c-c5600f1f4f94 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:14.726812 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 418/835] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Nov 2 03:22:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:22:14.732766 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf999639-3458-4895-b59c-c5600f1f4f94 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:14.805958 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 418/836] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Nov 2 03:22:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Nov 02 03:22:14.811745 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e126d7-7567-46fd-8689-e02a5c8a24dc None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:14.829367 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf999639-3458-4895-b59c-c5600f1f4f94 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:14.867414 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e97dd04-42e0-4200-8608-60634f7be058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:14.868045 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e97dd04-42e0-4200-8608-60634f7be058 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:14.868326 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e97dd04-42e0-4200-8608-60634f7be058 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:14.868597 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e97dd04-42e0-4200-8608-60634f7be058 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:14.869316 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e97dd04-42e0-4200-8608-60634f7be058 None admin] RBAC: Adding query filter params (role.id=c92260a73f5748618d4d4cbae46e5777, scope.project.id=d0b5e7017ac64ae6ab62c1a05aa54b72, user.id=44961b06151c4da382fcd54dd1f1e146) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:22:14.869904 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e97dd04-42e0-4200-8608-60634f7be058 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:14.871039 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e97dd04-42e0-4200-8608-60634f7be058 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:14.874074 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:14.877917 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 419/837] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Nov 2 03:22:14 2022] GET /identity/v3/role_assignments?user.id=44961b06151c4da382fcd54dd1f1e146&scope.project.id=d0b5e7017ac64ae6ab62c1a05aa54b72&role.id=c92260a73f5748618d4d4cbae46e5777 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:22:14.881460 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:14.916485 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e126d7-7567-46fd-8689-e02a5c8a24dc None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:14.953044 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84af022-201e-4328-8511-a2fcd875bb16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:14.953725 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84af022-201e-4328-8511-a2fcd875bb16 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:14.954002 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84af022-201e-4328-8511-a2fcd875bb16 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:14.954273 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84af022-201e-4328-8511-a2fcd875bb16 None admin] PATH_INFO: `/v3/projects/8f338522ad2f4f2692365f36de706fc0` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:14.955447 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84af022-201e-4328-8511-a2fcd875bb16 None admin] RBAC: Authorizing `identity:get_project(project_id=8f338522ad2f4f2692365f36de706fc0)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:14.956926 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84af022-201e-4328-8511-a2fcd875bb16 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:14.958636 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 419/838] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Nov 2 03:22:14 2022] GET /identity/v3/projects/8f338522ad2f4f2692365f36de706fc0 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:22:14.965111 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e97dd04-42e0-4200-8608-60634f7be058 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:15.072843 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e97dd04-42e0-4200-8608-60634f7be058 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:15.114537 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec117f9-0ee8-40b9-a678-d2c00e5618ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:15.115724 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec117f9-0ee8-40b9-a678-d2c00e5618ac None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:15.116042 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec117f9-0ee8-40b9-a678-d2c00e5618ac None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:15.116519 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec117f9-0ee8-40b9-a678-d2c00e5618ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:15.117620 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec117f9-0ee8-40b9-a678-d2c00e5618ac None admin] RBAC: Adding query filter params (role.id=addc0db6b4ac4362b07fe21e14f89180, scope.project.id=8f338522ad2f4f2692365f36de706fc0, user.id=459759940cc345dd9c9777e070288fc5) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:22:15.118024 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec117f9-0ee8-40b9-a678-d2c00e5618ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:15.119369 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec117f9-0ee8-40b9-a678-d2c00e5618ac None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:15.126647 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 420/839] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Nov 2 03:22:14 2022] GET /identity/v3/role_assignments?user.id=459759940cc345dd9c9777e070288fc5&scope.project.id=8f338522ad2f4f2692365f36de706fc0&role.id=addc0db6b4ac4362b07fe21e14f89180 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:22:17.469497 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a8bf7f-b510-4603-a1e7-e380d97b3826 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:17.470059 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a8bf7f-b510-4603-a1e7-e380d97b3826 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:17.470059 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a8bf7f-b510-4603-a1e7-e380d97b3826 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:17.471387 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 420/840] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:22:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:22:17.477386 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d2cbd-325d-4cff-931e-0daa54dd3b1c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:17.477777 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d2cbd-325d-4cff-931e-0daa54dd3b1c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:17.477938 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d2cbd-325d-4cff-931e-0daa54dd3b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:17.496799 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-4d1d2cbd-325d-4cff-931e-0daa54dd3b1c None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:22:17.497966 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4d1d2cbd-325d-4cff-931e-0daa54dd3b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:22:17.554683 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:17.563720 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 421/841] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:22:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 03:22:17.838745 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a8bf7f-b510-4603-a1e7-e380d97b3826 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:17.936845 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a8bf7f-b510-4603-a1e7-e380d97b3826 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:17.969689 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ab47b3-3c9e-4bf9-81ed-39b991cdb666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:17.970299 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab47b3-3c9e-4bf9-81ed-39b991cdb666 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:17.970573 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab47b3-3c9e-4bf9-81ed-39b991cdb666 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:17.970840 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab47b3-3c9e-4bf9-81ed-39b991cdb666 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:17.971918 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ab47b3-3c9e-4bf9-81ed-39b991cdb666 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:17.973356 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ab47b3-3c9e-4bf9-81ed-39b991cdb666 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:17.974891 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 421/842] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:22:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:22:17.980505 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1d2cbd-325d-4cff-931e-0daa54dd3b1c None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:18.074082 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1d2cbd-325d-4cff-931e-0daa54dd3b1c None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:18.106639 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d58d89-008e-4b27-81e6-cbfa84eaadf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:18.107171 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d58d89-008e-4b27-81e6-cbfa84eaadf6 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:18.107443 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d58d89-008e-4b27-81e6-cbfa84eaadf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:18.107710 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d58d89-008e-4b27-81e6-cbfa84eaadf6 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:18.108370 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d58d89-008e-4b27-81e6-cbfa84eaadf6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:18.109753 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d58d89-008e-4b27-81e6-cbfa84eaadf6 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:18.131124 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 422/843] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Nov 2 03:22:17 2022] POST /identity/v3/users => generated 286 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:22:20.276850 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429deb1b-e787-4275-bf4a-c6cd3ee5a7b8 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:20.277413 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429deb1b-e787-4275-bf4a-c6cd3ee5a7b8 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:20.277413 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429deb1b-e787-4275-bf4a-c6cd3ee5a7b8 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:20.278732 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 422/844] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:22:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:22:20.283861 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4713e6f-5bf9-428c-8418-48c1cadcd364 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:20.284297 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4713e6f-5bf9-428c-8418-48c1cadcd364 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:20.284430 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4713e6f-5bf9-428c-8418-48c1cadcd364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:20.303095 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d4713e6f-5bf9-428c-8418-48c1cadcd364 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:22:20.304312 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d4713e6f-5bf9-428c-8418-48c1cadcd364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:22:20.347198 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:20.369586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 423/845] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:22:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 03:22:20.634112 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429deb1b-e787-4275-bf4a-c6cd3ee5a7b8 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:20.642953 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:20.733337 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429deb1b-e787-4275-bf4a-c6cd3ee5a7b8 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:20.766252 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b649375-2171-494a-b9e5-20c630ede1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:20.766868 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b649375-2171-494a-b9e5-20c630ede1c6 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:20.767147 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b649375-2171-494a-b9e5-20c630ede1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:20.767418 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b649375-2171-494a-b9e5-20c630ede1c6 None admin] PATH_INFO: `/v3/roles/f25d67f31fee4a92b3060e45a3fbe379` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:20.768822 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b649375-2171-494a-b9e5-20c630ede1c6 None admin] RBAC: Authorizing `identity:get_role(role_id=f25d67f31fee4a92b3060e45a3fbe379)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:20.770172 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b649375-2171-494a-b9e5-20c630ede1c6 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:20.772034 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 423/846] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Nov 2 03:22:20 2022] GET /identity/v3/roles/f25d67f31fee4a92b3060e45a3fbe379 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:22:20.777717 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4713e6f-5bf9-428c-8418-48c1cadcd364 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:20.872555 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4713e6f-5bf9-428c-8418-48c1cadcd364 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:20.904902 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d0cc61-81a3-4893-aa0b-d34380d4a0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:20.905515 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d0cc61-81a3-4893-aa0b-d34380d4a0bc None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:20.905787 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d0cc61-81a3-4893-aa0b-d34380d4a0bc None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:20.906058 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d0cc61-81a3-4893-aa0b-d34380d4a0bc None admin] PATH_INFO: `/v3/users/e30c61ea162f4ee0a171a80847e6400d` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:20.919018 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d0cc61-81a3-4893-aa0b-d34380d4a0bc None admin] RBAC: Authorizing `identity:get_user(user_id=e30c61ea162f4ee0a171a80847e6400d)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:20.920469 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d0cc61-81a3-4893-aa0b-d34380d4a0bc None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:20.921996 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 424/847] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Nov 2 03:22:20 2022] GET /identity/v3/users/e30c61ea162f4ee0a171a80847e6400d => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:22:20.927556 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b649375-2171-494a-b9e5-20c630ede1c6 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:20.999294 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:21.024439 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b649375-2171-494a-b9e5-20c630ede1c6 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:21.056443 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dac8cfb-0231-4bab-8d83-e839f1b9f021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:21.057055 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac8cfb-0231-4bab-8d83-e839f1b9f021 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:21.057334 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac8cfb-0231-4bab-8d83-e839f1b9f021 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:21.057605 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dac8cfb-0231-4bab-8d83-e839f1b9f021 None admin] PATH_INFO: `/v3/projects/8f338522ad2f4f2692365f36de706fc0` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:21.058711 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dac8cfb-0231-4bab-8d83-e839f1b9f021 None admin] RBAC: Authorizing `identity:get_project(project_id=8f338522ad2f4f2692365f36de706fc0)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:21.060174 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dac8cfb-0231-4bab-8d83-e839f1b9f021 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:21.061766 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 424/848] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Nov 2 03:22:20 2022] GET /identity/v3/projects/8f338522ad2f4f2692365f36de706fc0 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:22:21.067540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d0cc61-81a3-4893-aa0b-d34380d4a0bc None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:21.106291 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:21.164532 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d0cc61-81a3-4893-aa0b-d34380d4a0bc None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:21.196296 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75285800-2e48-428e-8fcd-d3106f0f509e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:21.196944 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75285800-2e48-428e-8fcd-d3106f0f509e None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:21.197225 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75285800-2e48-428e-8fcd-d3106f0f509e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:21.197502 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75285800-2e48-428e-8fcd-d3106f0f509e None admin] PATH_INFO: `/v3/projects/8f338522ad2f4f2692365f36de706fc0/users/e30c61ea162f4ee0a171a80847e6400d/roles/f25d67f31fee4a92b3060e45a3fbe379` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:21.199606 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75285800-2e48-428e-8fcd-d3106f0f509e None admin] RBAC: Authorizing `identity:create_grant(project_id=8f338522ad2f4f2692365f36de706fc0, user_id=e30c61ea162f4ee0a171a80847e6400d, role_id=f25d67f31fee4a92b3060e45a3fbe379)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:21.201058 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75285800-2e48-428e-8fcd-d3106f0f509e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:21.213839 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 425/849] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Nov 2 03:22:21 2022] PUT /identity/v3/projects/8f338522ad2f4f2692365f36de706fc0/users/e30c61ea162f4ee0a171a80847e6400d/roles/f25d67f31fee4a92b3060e45a3fbe379 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 03:22:23.315714 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d622f9-a01f-4bed-ac63-4873a54d9c94 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:23.316287 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d622f9-a01f-4bed-ac63-4873a54d9c94 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:23.316287 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d622f9-a01f-4bed-ac63-4873a54d9c94 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:23.317611 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 425/850] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:22:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:22:23.323230 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c0cbf3-6efc-4541-9b0a-2083acdaa8d1 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:23.323671 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c0cbf3-6efc-4541-9b0a-2083acdaa8d1 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:23.323817 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c0cbf3-6efc-4541-9b0a-2083acdaa8d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:23.342197 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-42c0cbf3-6efc-4541-9b0a-2083acdaa8d1 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:22:23.343379 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-42c0cbf3-6efc-4541-9b0a-2083acdaa8d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:22:23.408124 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 426/851] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:22:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 03:22:23.672548 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d622f9-a01f-4bed-ac63-4873a54d9c94 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:23.771901 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d622f9-a01f-4bed-ac63-4873a54d9c94 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:23.804962 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60be6d66-15d5-4b83-a2a5-0ad782967acf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:23.805571 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60be6d66-15d5-4b83-a2a5-0ad782967acf None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:23.805847 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60be6d66-15d5-4b83-a2a5-0ad782967acf None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:23.806115 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60be6d66-15d5-4b83-a2a5-0ad782967acf None admin] PATH_INFO: `/v3/roles/f25d67f31fee4a92b3060e45a3fbe379` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:23.807475 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60be6d66-15d5-4b83-a2a5-0ad782967acf None admin] RBAC: Authorizing `identity:get_role(role_id=f25d67f31fee4a92b3060e45a3fbe379)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:23.808851 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60be6d66-15d5-4b83-a2a5-0ad782967acf None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:23.810140 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 426/852] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Nov 2 03:22:23 2022] GET /identity/v3/roles/f25d67f31fee4a92b3060e45a3fbe379 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:22:23.815921 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c0cbf3-6efc-4541-9b0a-2083acdaa8d1 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:23.842487 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:23.911979 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42c0cbf3-6efc-4541-9b0a-2083acdaa8d1 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:23.944642 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c30c9f-1537-4969-a6b3-7bc8f750561b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:23.945202 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c30c9f-1537-4969-a6b3-7bc8f750561b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:23.945480 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c30c9f-1537-4969-a6b3-7bc8f750561b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:23.945750 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c30c9f-1537-4969-a6b3-7bc8f750561b None admin] PATH_INFO: `/v3/users/e30c61ea162f4ee0a171a80847e6400d` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:23.946844 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c30c9f-1537-4969-a6b3-7bc8f750561b None admin] RBAC: Authorizing `identity:get_user(user_id=e30c61ea162f4ee0a171a80847e6400d)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:23.948275 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c30c9f-1537-4969-a6b3-7bc8f750561b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:23.949807 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 427/853] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Nov 2 03:22:23 2022] GET /identity/v3/users/e30c61ea162f4ee0a171a80847e6400d => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:22:23.955367 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60be6d66-15d5-4b83-a2a5-0ad782967acf None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:23.967217 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:24.053287 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60be6d66-15d5-4b83-a2a5-0ad782967acf None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:24.085682 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a6ee9a-ec2e-4088-a4f4-436000e565bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:24.086274 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6ee9a-ec2e-4088-a4f4-436000e565bd None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:24.086562 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6ee9a-ec2e-4088-a4f4-436000e565bd None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:24.086836 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a6ee9a-ec2e-4088-a4f4-436000e565bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:24.152506 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a6ee9a-ec2e-4088-a4f4-436000e565bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:24.153935 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a6ee9a-ec2e-4088-a4f4-436000e565bd None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:24.165013 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:24.249224 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 427/854] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Nov 2 03:22:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 03:22:24.254989 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c30c9f-1537-4969-a6b3-7bc8f750561b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:24.344343 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:24.350751 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c30c9f-1537-4969-a6b3-7bc8f750561b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:24.382787 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8627cfa6-d916-4d7c-9b83-d712868b2d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:24.383398 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627cfa6-d916-4d7c-9b83-d712868b2d55 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:24.383674 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627cfa6-d916-4d7c-9b83-d712868b2d55 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:24.383943 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8627cfa6-d916-4d7c-9b83-d712868b2d55 None admin] PATH_INFO: `/v3/projects/8f338522ad2f4f2692365f36de706fc0` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:24.385060 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8627cfa6-d916-4d7c-9b83-d712868b2d55 None admin] RBAC: Authorizing `identity:get_project(project_id=8f338522ad2f4f2692365f36de706fc0)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:24.386463 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8627cfa6-d916-4d7c-9b83-d712868b2d55 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:24.388055 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 428/855] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Nov 2 03:22:24 2022] GET /identity/v3/projects/8f338522ad2f4f2692365f36de706fc0 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:22:24.394246 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a6ee9a-ec2e-4088-a4f4-436000e565bd None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:24.490881 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a6ee9a-ec2e-4088-a4f4-436000e565bd None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:24.501240 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:24.525069 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a2747e-bfc5-43cc-93c0-0a2d2e6fcf13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:24.525674 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a2747e-bfc5-43cc-93c0-0a2d2e6fcf13 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:24.525955 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a2747e-bfc5-43cc-93c0-0a2d2e6fcf13 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:24.526233 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a2747e-bfc5-43cc-93c0-0a2d2e6fcf13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:24.526953 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a2747e-bfc5-43cc-93c0-0a2d2e6fcf13 None admin] RBAC: Adding query filter params (role.id=f25d67f31fee4a92b3060e45a3fbe379, scope.project.id=8f338522ad2f4f2692365f36de706fc0, user.id=e30c61ea162f4ee0a171a80847e6400d) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:22:24.527259 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a2747e-bfc5-43cc-93c0-0a2d2e6fcf13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:24.528705 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a2747e-bfc5-43cc-93c0-0a2d2e6fcf13 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:24.534567 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 428/856] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Nov 2 03:22:24 2022] GET /identity/v3/role_assignments?user.id=e30c61ea162f4ee0a171a80847e6400d&scope.project.id=8f338522ad2f4f2692365f36de706fc0&role.id=f25d67f31fee4a92b3060e45a3fbe379 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:22:26.666341 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249067d5-d8c5-4ba6-ada6-a0c2e2163dfd None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:26.666862 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249067d5-d8c5-4ba6-ada6-a0c2e2163dfd None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:26.666862 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249067d5-d8c5-4ba6-ada6-a0c2e2163dfd None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:26.668254 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 429/857] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:22:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:22:26.673476 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0e73a2-7dc2-4e04-8f80-06a6c2590b8a None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:26.673852 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0e73a2-7dc2-4e04-8f80-06a6c2590b8a None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:26.674037 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0e73a2-7dc2-4e04-8f80-06a6c2590b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:26.691881 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-2b0e73a2-7dc2-4e04-8f80-06a6c2590b8a None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:22:26.693158 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-2b0e73a2-7dc2-4e04-8f80-06a6c2590b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:22:26.757733 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 429/858] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:22:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 03:22:27.023747 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249067d5-d8c5-4ba6-ada6-a0c2e2163dfd None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:27.120930 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249067d5-d8c5-4ba6-ada6-a0c2e2163dfd None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:27.153047 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e5d5db-2467-4b95-97c1-0018b0a37232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:27.153659 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e5d5db-2467-4b95-97c1-0018b0a37232 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:27.153941 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e5d5db-2467-4b95-97c1-0018b0a37232 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:27.154212 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e5d5db-2467-4b95-97c1-0018b0a37232 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:27.155306 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e5d5db-2467-4b95-97c1-0018b0a37232 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:27.156745 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e5d5db-2467-4b95-97c1-0018b0a37232 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:27.158256 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 430/859] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:22:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:22:27.163980 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0e73a2-7dc2-4e04-8f80-06a6c2590b8a None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:27.260387 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0e73a2-7dc2-4e04-8f80-06a6c2590b8a None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:27.262599 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:27.293428 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca23ee59-8600-47f4-8036-4ca55bbe531a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:27.293966 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23ee59-8600-47f4-8036-4ca55bbe531a None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:27.294241 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23ee59-8600-47f4-8036-4ca55bbe531a None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:27.294509 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca23ee59-8600-47f4-8036-4ca55bbe531a None admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:27.295141 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca23ee59-8600-47f4-8036-4ca55bbe531a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:27.296595 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca23ee59-8600-47f4-8036-4ca55bbe531a None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:27.322300 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 430/860] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Nov 2 03:22:27 2022] POST /identity/v3/projects => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:22:29.484358 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b22faa7-185f-44fc-89a0-bb907941e777 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:29.484907 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b22faa7-185f-44fc-89a0-bb907941e777 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:29.484907 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b22faa7-185f-44fc-89a0-bb907941e777 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:29.486212 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 431/861] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:22:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:22:29.492727 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38928f61-c9df-4945-9c20-5aec49196d6d None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:29.493143 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38928f61-c9df-4945-9c20-5aec49196d6d None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:29.493280 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38928f61-c9df-4945-9c20-5aec49196d6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:29.511812 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-38928f61-c9df-4945-9c20-5aec49196d6d None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:22:29.513035 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-38928f61-c9df-4945-9c20-5aec49196d6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:22:29.577947 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 431/862] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:22:29 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 03:22:29.843344 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b22faa7-185f-44fc-89a0-bb907941e777 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:29.874441 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:29.941354 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b22faa7-185f-44fc-89a0-bb907941e777 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:29.972813 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a10573-ca83-4d11-bb75-a50659f072d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:29.973419 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a10573-ca83-4d11-bb75-a50659f072d5 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:29.973694 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a10573-ca83-4d11-bb75-a50659f072d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:29.974472 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a10573-ca83-4d11-bb75-a50659f072d5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:29.975559 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a10573-ca83-4d11-bb75-a50659f072d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:29.976990 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a10573-ca83-4d11-bb75-a50659f072d5 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:29.978496 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 432/863] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:22:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:22:29.984113 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38928f61-c9df-4945-9c20-5aec49196d6d None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:30.065633 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:30.080548 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38928f61-c9df-4945-9c20-5aec49196d6d None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:30.113282 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784dcc71-42f5-4369-8d99-774cea91792f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:30.113825 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784dcc71-42f5-4369-8d99-774cea91792f None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:30.114102 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784dcc71-42f5-4369-8d99-774cea91792f None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:30.114374 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784dcc71-42f5-4369-8d99-774cea91792f None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:30.115559 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784dcc71-42f5-4369-8d99-774cea91792f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:30.117007 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784dcc71-42f5-4369-8d99-774cea91792f None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:30.137034 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 432/864] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Nov 2 03:22:29 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 03:22:32.277615 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3443abd0-4ab3-4354-a577-5bbfa4785033 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:32.278209 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3443abd0-4ab3-4354-a577-5bbfa4785033 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:32.278209 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3443abd0-4ab3-4354-a577-5bbfa4785033 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:32.279481 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 433/865] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:22:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:22:32.284694 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48990b5-3641-4f8c-ac1e-4066397de417 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:32.285098 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48990b5-3641-4f8c-ac1e-4066397de417 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:32.285244 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48990b5-3641-4f8c-ac1e-4066397de417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:32.302909 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-d48990b5-3641-4f8c-ac1e-4066397de417 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:22:32.304119 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d48990b5-3641-4f8c-ac1e-4066397de417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:22:32.369373 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 433/866] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:22:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 03:22:32.640253 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3443abd0-4ab3-4354-a577-5bbfa4785033 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:32.734524 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:32.738890 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3443abd0-4ab3-4354-a577-5bbfa4785033 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:32.772242 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702452a1-307f-4dcc-b6ae-ab3898083c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:32.772860 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702452a1-307f-4dcc-b6ae-ab3898083c10 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:32.773148 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702452a1-307f-4dcc-b6ae-ab3898083c10 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:32.773422 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702452a1-307f-4dcc-b6ae-ab3898083c10 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:32.783151 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702452a1-307f-4dcc-b6ae-ab3898083c10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:22:32.788442 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-702452a1-307f-4dcc-b6ae-ab3898083c10 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 03:22:32.788442 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:22:32.788442 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:22:32.788442 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:22:32.788442 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:22:32.788442 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:22:32.788442 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:22:32.788442 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:22:32.788442 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:22:32.788442 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:22:32.788442 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:22:32.788442 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:22:32.788442 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:22:32.788442 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:22:32.788442 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:22:32.788442 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:22:32.789523 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:22:32.789523 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:22:32.789523 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:22:32.789523 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:22:32.789523 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:22:32.789523 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:22:32.789523 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:22:32.789523 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:22:32.789523 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:22:32.789523 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:22:32.789523 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:22:32.789523 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:22:32.789523 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:22:32.789523 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:22:32.789523 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:22:32.789523 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:22:32.790555 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:22:32.790555 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:22:32.790555 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:22:32.790555 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:22:32.790555 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:22:32.790555 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:22:32.790555 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:22:32.790555 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:22:32.790555 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:22:32.790555 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:22:32.790555 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 03:22:32.790555 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:22:32.790555 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 434/867] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Nov 2 03:22:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:22:32.795444 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48990b5-3641-4f8c-ac1e-4066397de417 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:32.863414 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:32.892592 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48990b5-3641-4f8c-ac1e-4066397de417 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:32.926487 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9becefb-c7d0-496d-af75-298a23969da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:32.927641 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9becefb-c7d0-496d-af75-298a23969da1 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:32.927916 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9becefb-c7d0-496d-af75-298a23969da1 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:32.928216 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9becefb-c7d0-496d-af75-298a23969da1 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:32.928871 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9becefb-c7d0-496d-af75-298a23969da1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:22:32.929171 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9becefb-c7d0-496d-af75-298a23969da1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:32.930519 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9becefb-c7d0-496d-af75-298a23969da1 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:32.938346 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 434/868] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Nov 2 03:22:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:22:32.944049 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702452a1-307f-4dcc-b6ae-ab3898083c10 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:33.040969 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702452a1-307f-4dcc-b6ae-ab3898083c10 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:33.073393 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a533f2-5511-4a61-91e7-db8f14e6ed22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:33.074003 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a533f2-5511-4a61-91e7-db8f14e6ed22 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:33.074281 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a533f2-5511-4a61-91e7-db8f14e6ed22 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:33.074551 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a533f2-5511-4a61-91e7-db8f14e6ed22 None admin] PATH_INFO: `/v3/users/cc4ef3fa613c4563a70862446a7b8d5e` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:33.086938 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a533f2-5511-4a61-91e7-db8f14e6ed22 None admin] RBAC: Authorizing `identity:get_user(user_id=cc4ef3fa613c4563a70862446a7b8d5e)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:33.088391 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a533f2-5511-4a61-91e7-db8f14e6ed22 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:33.089937 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 435/869] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Nov 2 03:22:32 2022] GET /identity/v3/users/cc4ef3fa613c4563a70862446a7b8d5e => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:22:33.095444 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9becefb-c7d0-496d-af75-298a23969da1 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:33.191807 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9becefb-c7d0-496d-af75-298a23969da1 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:33.224601 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097303f1-2e20-443a-a33d-5e0442a708ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:33.225210 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097303f1-2e20-443a-a33d-5e0442a708ab None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:33.225488 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097303f1-2e20-443a-a33d-5e0442a708ab None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:33.225761 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097303f1-2e20-443a-a33d-5e0442a708ab None admin] PATH_INFO: `/v3/projects/b205481227d7479a86201072e9b210fb` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:33.226862 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097303f1-2e20-443a-a33d-5e0442a708ab None admin] RBAC: Authorizing `identity:get_project(project_id=b205481227d7479a86201072e9b210fb)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:33.228294 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097303f1-2e20-443a-a33d-5e0442a708ab None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:33.229867 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 435/870] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Nov 2 03:22:33 2022] GET /identity/v3/projects/b205481227d7479a86201072e9b210fb => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:22:33.235818 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a533f2-5511-4a61-91e7-db8f14e6ed22 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:33.254235 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:33.331583 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a533f2-5511-4a61-91e7-db8f14e6ed22 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:33.365101 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd213bc-a7e1-4393-9396-3760f252bfa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:33.365744 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd213bc-a7e1-4393-9396-3760f252bfa6 None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:33.366025 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd213bc-a7e1-4393-9396-3760f252bfa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:33.366298 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd213bc-a7e1-4393-9396-3760f252bfa6 None admin] PATH_INFO: `/v3/projects/b205481227d7479a86201072e9b210fb/users/cc4ef3fa613c4563a70862446a7b8d5e/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:33.368199 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd213bc-a7e1-4393-9396-3760f252bfa6 None admin] RBAC: Authorizing `identity:create_grant(project_id=b205481227d7479a86201072e9b210fb, user_id=cc4ef3fa613c4563a70862446a7b8d5e, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:33.369611 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd213bc-a7e1-4393-9396-3760f252bfa6 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:33.381505 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 436/871] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Nov 2 03:22:33 2022] PUT /identity/v3/projects/b205481227d7479a86201072e9b210fb/users/cc4ef3fa613c4563a70862446a7b8d5e/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:22:35.461209 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f8783-f95e-463e-b9e2-42acc6cb050d None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:35.461739 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f8783-f95e-463e-b9e2-42acc6cb050d None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:35.461739 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0f8783-f95e-463e-b9e2-42acc6cb050d None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:35.463073 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 436/872] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:22:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:22:35.468230 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ffb4c6-fc59-48f1-a563-b66f1842d85c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:35.468661 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ffb4c6-fc59-48f1-a563-b66f1842d85c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:35.468784 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ffb4c6-fc59-48f1-a563-b66f1842d85c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:35.486933 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-00ffb4c6-fc59-48f1-a563-b66f1842d85c None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:22:35.488394 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-00ffb4c6-fc59-48f1-a563-b66f1842d85c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:22:35.552074 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 437/873] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:22:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 03:22:35.818037 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0f8783-f95e-463e-b9e2-42acc6cb050d None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:35.820864 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:35.917580 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0f8783-f95e-463e-b9e2-42acc6cb050d None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:35.949333 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97416e5-69e4-4271-b908-2e4093648186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:35.949940 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97416e5-69e4-4271-b908-2e4093648186 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:35.950212 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97416e5-69e4-4271-b908-2e4093648186 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:35.950479 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97416e5-69e4-4271-b908-2e4093648186 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:36.074414 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97416e5-69e4-4271-b908-2e4093648186 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:22:36.079176 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-f97416e5-69e4-4271-b908-2e4093648186 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 03:22:36.079176 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:22:36.079176 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:22:36.079176 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:22:36.079176 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:22:36.079176 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:22:36.079176 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:22:36.079176 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:22:36.079176 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:22:36.079176 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:22:36.079176 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:22:36.079176 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:22:36.079176 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:22:36.079176 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:22:36.079176 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:22:36.079176 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:22:36.080292 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:22:36.080292 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:22:36.080292 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:22:36.080292 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:22:36.080292 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:22:36.080292 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:22:36.080292 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:22:36.080292 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:22:36.080292 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:22:36.080292 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:22:36.080292 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:22:36.080292 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:22:36.080292 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:22:36.080292 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:22:36.080292 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:22:36.080292 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:22:36.081176 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:22:36.081176 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:22:36.081176 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:22:36.081176 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:22:36.081176 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:22:36.081176 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:22:36.081176 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:22:36.081176 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:22:36.081176 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:22:36.081176 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:22:36.081176 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 03:22:36.081176 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:22:36.081176 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 437/874] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Nov 2 03:22:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:22:36.086467 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ffb4c6-fc59-48f1-a563-b66f1842d85c None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:36.096531 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:36.184347 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ffb4c6-fc59-48f1-a563-b66f1842d85c None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:36.217116 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b51e6-70db-4c1e-bf1e-f2e52f5c00f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:36.217719 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b51e6-70db-4c1e-bf1e-f2e52f5c00f7 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:36.217994 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b51e6-70db-4c1e-bf1e-f2e52f5c00f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:36.218263 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87b51e6-70db-4c1e-bf1e-f2e52f5c00f7 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:36.218944 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87b51e6-70db-4c1e-bf1e-f2e52f5c00f7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:22:36.219245 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87b51e6-70db-4c1e-bf1e-f2e52f5c00f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:36.220646 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87b51e6-70db-4c1e-bf1e-f2e52f5c00f7 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:36.229016 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 438/875] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Nov 2 03:22:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:22:36.234969 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97416e5-69e4-4271-b908-2e4093648186 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:36.294712 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:36.331597 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97416e5-69e4-4271-b908-2e4093648186 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:36.364382 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5612f7e1-67b3-4265-9ec0-d7038425f8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:36.364994 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5612f7e1-67b3-4265-9ec0-d7038425f8ed None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:36.365274 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5612f7e1-67b3-4265-9ec0-d7038425f8ed None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:36.365547 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5612f7e1-67b3-4265-9ec0-d7038425f8ed None admin] PATH_INFO: `/v3/users/cc4ef3fa613c4563a70862446a7b8d5e` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:36.366909 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5612f7e1-67b3-4265-9ec0-d7038425f8ed None admin] RBAC: Authorizing `identity:get_user(user_id=cc4ef3fa613c4563a70862446a7b8d5e)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:36.368355 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5612f7e1-67b3-4265-9ec0-d7038425f8ed None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:36.369895 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 438/876] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Nov 2 03:22:36 2022] GET /identity/v3/users/cc4ef3fa613c4563a70862446a7b8d5e => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:22:36.375542 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b51e6-70db-4c1e-bf1e-f2e52f5c00f7 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:36.441327 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:36.471092 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87b51e6-70db-4c1e-bf1e-f2e52f5c00f7 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:36.502903 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f22ef5-eb88-454a-bbbc-663278779f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:36.503488 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f22ef5-eb88-454a-bbbc-663278779f1c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:36.503761 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f22ef5-eb88-454a-bbbc-663278779f1c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:36.504056 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f22ef5-eb88-454a-bbbc-663278779f1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:36.568470 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f22ef5-eb88-454a-bbbc-663278779f1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:36.569901 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f22ef5-eb88-454a-bbbc-663278779f1c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:36.634043 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:36.664643 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 439/877] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Nov 2 03:22:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 03:22:36.670792 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5612f7e1-67b3-4265-9ec0-d7038425f8ed None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:36.766430 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5612f7e1-67b3-4265-9ec0-d7038425f8ed None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:36.793754 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:36.800010 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1d47ee-b3d6-4606-9458-ca8cd44ce43b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:36.800636 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d47ee-b3d6-4606-9458-ca8cd44ce43b None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:36.800917 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d47ee-b3d6-4606-9458-ca8cd44ce43b None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:36.801189 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1d47ee-b3d6-4606-9458-ca8cd44ce43b None admin] PATH_INFO: `/v3/projects/b205481227d7479a86201072e9b210fb` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:36.802318 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1d47ee-b3d6-4606-9458-ca8cd44ce43b None admin] RBAC: Authorizing `identity:get_project(project_id=b205481227d7479a86201072e9b210fb)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:36.803725 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1d47ee-b3d6-4606-9458-ca8cd44ce43b None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:36.805360 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 439/878] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Nov 2 03:22:36 2022] GET /identity/v3/projects/b205481227d7479a86201072e9b210fb => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:22:36.811472 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f22ef5-eb88-454a-bbbc-663278779f1c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:36.907172 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f22ef5-eb88-454a-bbbc-663278779f1c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:36.939548 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf6043f-230f-4841-999a-426019967a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:36.940168 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6043f-230f-4841-999a-426019967a4c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:36.940448 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6043f-230f-4841-999a-426019967a4c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:36.940719 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf6043f-230f-4841-999a-426019967a4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:36.941434 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf6043f-230f-4841-999a-426019967a4c None admin] RBAC: Adding query filter params (role.id=addc0db6b4ac4362b07fe21e14f89180, scope.project.id=b205481227d7479a86201072e9b210fb, user.id=cc4ef3fa613c4563a70862446a7b8d5e) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:22:36.941735 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf6043f-230f-4841-999a-426019967a4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:36.943683 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf6043f-230f-4841-999a-426019967a4c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:36.949238 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 440/879] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Nov 2 03:22:36 2022] GET /identity/v3/role_assignments?user.id=cc4ef3fa613c4563a70862446a7b8d5e&scope.project.id=b205481227d7479a86201072e9b210fb&role.id=addc0db6b4ac4362b07fe21e14f89180 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:22:39.101149 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e68518f-c726-4e06-8529-37fb711cfc44 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:39.101790 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e68518f-c726-4e06-8529-37fb711cfc44 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:39.101790 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e68518f-c726-4e06-8529-37fb711cfc44 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:39.103016 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 440/880] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:22:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:22:39.108187 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f291a731-6084-4fa1-be14-4bf5f02416b6 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:39.108591 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f291a731-6084-4fa1-be14-4bf5f02416b6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:39.108744 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f291a731-6084-4fa1-be14-4bf5f02416b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:39.127202 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-f291a731-6084-4fa1-be14-4bf5f02416b6 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:22:39.128430 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f291a731-6084-4fa1-be14-4bf5f02416b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:22:39.154590 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:39.194182 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 441/881] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:22:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 03:22:39.463477 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e68518f-c726-4e06-8529-37fb711cfc44 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:39.562203 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e68518f-c726-4e06-8529-37fb711cfc44 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:39.595123 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c23f74-c9ca-4ea1-bef7-e76faecda26e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:39.595716 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c23f74-c9ca-4ea1-bef7-e76faecda26e None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:39.596028 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c23f74-c9ca-4ea1-bef7-e76faecda26e None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:39.596298 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c23f74-c9ca-4ea1-bef7-e76faecda26e None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:39.596963 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c23f74-c9ca-4ea1-bef7-e76faecda26e None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:39.598301 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c23f74-c9ca-4ea1-bef7-e76faecda26e None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:39.613551 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 441/882] 10.222.0.26 () {60 vars in 1138 bytes} [Wed Nov 2 03:22:39 2022] POST /identity/v3/domains => generated 267 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:22:41.734821 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d39f1b5-9c31-4dab-9460-c835846ade29 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:41.735339 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d39f1b5-9c31-4dab-9460-c835846ade29 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:41.735339 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d39f1b5-9c31-4dab-9460-c835846ade29 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:41.736737 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 442/883] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:22:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:22:41.742112 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607d74e3-5da3-4dea-a4f2-7bbb07936d0f None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:41.742496 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607d74e3-5da3-4dea-a4f2-7bbb07936d0f None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:41.742657 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607d74e3-5da3-4dea-a4f2-7bbb07936d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:41.760692 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-607d74e3-5da3-4dea-a4f2-7bbb07936d0f None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:22:41.761860 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-607d74e3-5da3-4dea-a4f2-7bbb07936d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:22:41.812326 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:41.826400 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 442/884] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:22:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 03:22:42.091509 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d39f1b5-9c31-4dab-9460-c835846ade29 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:42.188818 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d39f1b5-9c31-4dab-9460-c835846ade29 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:42.222092 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79832bbc-c20b-4645-a0de-1fbf8271e3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:42.222709 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79832bbc-c20b-4645-a0de-1fbf8271e3e8 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:42.222988 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79832bbc-c20b-4645-a0de-1fbf8271e3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:42.223261 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79832bbc-c20b-4645-a0de-1fbf8271e3e8 None admin] PATH_INFO: `/v3/domains/724324dc49c64eba8819af9fd1c53030` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:42.234086 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79832bbc-c20b-4645-a0de-1fbf8271e3e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=724324dc49c64eba8819af9fd1c53030)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:42.235515 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79832bbc-c20b-4645-a0de-1fbf8271e3e8 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:42.237120 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 443/885] 10.222.0.26 () {56 vars in 1183 bytes} [Wed Nov 2 03:22:42 2022] GET /identity/v3/domains/724324dc49c64eba8819af9fd1c53030 => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:22:42.243138 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607d74e3-5da3-4dea-a4f2-7bbb07936d0f None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:42.337801 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607d74e3-5da3-4dea-a4f2-7bbb07936d0f None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:42.371166 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cebb8a-b1bc-4fe9-ab94-278e4d21df31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:42.371704 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cebb8a-b1bc-4fe9-ab94-278e4d21df31 None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:42.372004 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cebb8a-b1bc-4fe9-ab94-278e4d21df31 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:42.372280 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cebb8a-b1bc-4fe9-ab94-278e4d21df31 None admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:42.372916 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cebb8a-b1bc-4fe9-ab94-278e4d21df31 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:42.374323 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cebb8a-b1bc-4fe9-ab94-278e4d21df31 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:42.399886 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 443/886] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Nov 2 03:22:42 2022] POST /identity/v3/projects => generated 361 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:22:44.543040 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93910baf-ec62-411a-92b5-cf33365f0ea3 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:44.543548 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93910baf-ec62-411a-92b5-cf33365f0ea3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:44.543635 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93910baf-ec62-411a-92b5-cf33365f0ea3 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:44.545093 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 444/887] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:22:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:22:44.550454 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5590fbd-d441-4da6-bd28-2ca9a611b6c2 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:44.550879 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5590fbd-d441-4da6-bd28-2ca9a611b6c2 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:44.551025 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5590fbd-d441-4da6-bd28-2ca9a611b6c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:44.570234 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-d5590fbd-d441-4da6-bd28-2ca9a611b6c2 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:22:44.571413 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d5590fbd-d441-4da6-bd28-2ca9a611b6c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:22:44.605025 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:44.638570 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 444/888] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:22:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 03:22:44.905418 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93910baf-ec62-411a-92b5-cf33365f0ea3 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:44.931003 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:45.004270 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93910baf-ec62-411a-92b5-cf33365f0ea3 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:45.036466 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee85ddc5-b010-4297-bd48-51035e95bb45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:45.037074 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee85ddc5-b010-4297-bd48-51035e95bb45 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:45.037349 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee85ddc5-b010-4297-bd48-51035e95bb45 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:45.037616 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee85ddc5-b010-4297-bd48-51035e95bb45 None admin] PATH_INFO: `/v3/domains/724324dc49c64eba8819af9fd1c53030` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:45.038699 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee85ddc5-b010-4297-bd48-51035e95bb45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=724324dc49c64eba8819af9fd1c53030)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:45.040135 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee85ddc5-b010-4297-bd48-51035e95bb45 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:45.041665 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 445/889] 10.222.0.26 () {56 vars in 1183 bytes} [Wed Nov 2 03:22:44 2022] GET /identity/v3/domains/724324dc49c64eba8819af9fd1c53030 => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:22:45.047298 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5590fbd-d441-4da6-bd28-2ca9a611b6c2 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:45.143081 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5590fbd-d441-4da6-bd28-2ca9a611b6c2 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:45.176698 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d83ed75-dbd8-42d6-83db-84ca8fd8028a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:45.177237 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83ed75-dbd8-42d6-83db-84ca8fd8028a None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:45.177552 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83ed75-dbd8-42d6-83db-84ca8fd8028a None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:45.177826 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d83ed75-dbd8-42d6-83db-84ca8fd8028a None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:45.178476 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d83ed75-dbd8-42d6-83db-84ca8fd8028a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:45.179877 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d83ed75-dbd8-42d6-83db-84ca8fd8028a None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:45.199955 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 445/890] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Nov 2 03:22:45 2022] POST /identity/v3/users => generated 311 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:22:47.358177 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7326abba-801e-4560-80a0-e381e1db4bda None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:47.358731 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7326abba-801e-4560-80a0-e381e1db4bda None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:47.358731 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7326abba-801e-4560-80a0-e381e1db4bda None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:47.360081 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 446/891] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:22:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:22:47.365479 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc74df-e9af-422e-8d1b-f10b35757b8b None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:47.365857 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc74df-e9af-422e-8d1b-f10b35757b8b None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:47.366097 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abc74df-e9af-422e-8d1b-f10b35757b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:47.383734 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-6abc74df-e9af-422e-8d1b-f10b35757b8b None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:22:47.384942 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-6abc74df-e9af-422e-8d1b-f10b35757b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:22:47.404818 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:47.450587 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 446/892] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:22:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 03:22:47.719184 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7326abba-801e-4560-80a0-e381e1db4bda None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:47.809569 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:47.819143 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7326abba-801e-4560-80a0-e381e1db4bda None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:47.851138 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf51ca0-fa5e-44e4-99b7-ecebfb49ed89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:47.851758 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf51ca0-fa5e-44e4-99b7-ecebfb49ed89 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:47.852068 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf51ca0-fa5e-44e4-99b7-ecebfb49ed89 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:47.852344 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf51ca0-fa5e-44e4-99b7-ecebfb49ed89 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:47.863248 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf51ca0-fa5e-44e4-99b7-ecebfb49ed89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:22:47.868560 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-0bf51ca0-fa5e-44e4-99b7-ecebfb49ed89 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 03:22:47.868560 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:22:47.868560 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:22:47.868560 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:22:47.868560 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:22:47.868560 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:22:47.868560 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:22:47.868560 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:22:47.868560 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:22:47.868560 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:22:47.868560 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:22:47.868560 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:22:47.868560 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:22:47.868560 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:22:47.868560 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:22:47.868560 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:22:47.869540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:22:47.869540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:22:47.869540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:22:47.869540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:22:47.869540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:22:47.869540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:22:47.869540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:22:47.869540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:22:47.869540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:22:47.869540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:22:47.869540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:22:47.869540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:22:47.869540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:22:47.869540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:22:47.869540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:22:47.869540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:22:47.870455 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:22:47.870455 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:22:47.870455 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:22:47.870455 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:22:47.870455 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:22:47.870455 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:22:47.870455 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:22:47.870455 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:22:47.870455 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:22:47.870455 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:22:47.870455 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 03:22:47.870455 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:22:47.870455 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 447/893] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Nov 2 03:22:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 03:22:47.875674 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abc74df-e9af-422e-8d1b-f10b35757b8b None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:47.969356 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abc74df-e9af-422e-8d1b-f10b35757b8b None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:48.002363 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d72d0f-4cd9-44a7-9757-ae8ee33ab807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:48.002956 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d72d0f-4cd9-44a7-9757-ae8ee33ab807 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:48.003231 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d72d0f-4cd9-44a7-9757-ae8ee33ab807 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:48.003500 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d72d0f-4cd9-44a7-9757-ae8ee33ab807 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:48.004179 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d72d0f-4cd9-44a7-9757-ae8ee33ab807 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:22:48.004476 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d72d0f-4cd9-44a7-9757-ae8ee33ab807 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:48.005822 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d72d0f-4cd9-44a7-9757-ae8ee33ab807 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:48.013447 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 447/894] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Nov 2 03:22:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:22:48.019245 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf51ca0-fa5e-44e4-99b7-ecebfb49ed89 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:48.116324 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf51ca0-fa5e-44e4-99b7-ecebfb49ed89 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:48.148435 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9de355f-b7e2-4365-ae79-e36b625a7540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:48.149052 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de355f-b7e2-4365-ae79-e36b625a7540 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:48.149333 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de355f-b7e2-4365-ae79-e36b625a7540 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:48.149607 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de355f-b7e2-4365-ae79-e36b625a7540 None admin] PATH_INFO: `/v3/users/40c9ce5c302e4190b9ad14581f349f40` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:48.162597 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9de355f-b7e2-4365-ae79-e36b625a7540 None admin] RBAC: Authorizing `identity:get_user(user_id=40c9ce5c302e4190b9ad14581f349f40)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:48.164058 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9de355f-b7e2-4365-ae79-e36b625a7540 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:48.165607 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 448/895] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Nov 2 03:22:48 2022] GET /identity/v3/users/40c9ce5c302e4190b9ad14581f349f40 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:22:48.171124 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d72d0f-4cd9-44a7-9757-ae8ee33ab807 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:48.189683 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:48.267694 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d72d0f-4cd9-44a7-9757-ae8ee33ab807 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:48.300264 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51f0ed5-c93a-4f18-8d9f-600e6112afac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:48.300881 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f0ed5-c93a-4f18-8d9f-600e6112afac None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:48.301160 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f0ed5-c93a-4f18-8d9f-600e6112afac None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:48.301463 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51f0ed5-c93a-4f18-8d9f-600e6112afac None admin] PATH_INFO: `/v3/projects/f2ca43aa6a1042f08c4359d0cffed7b3` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:48.302575 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51f0ed5-c93a-4f18-8d9f-600e6112afac None admin] RBAC: Authorizing `identity:get_project(project_id=f2ca43aa6a1042f08c4359d0cffed7b3)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:48.304002 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51f0ed5-c93a-4f18-8d9f-600e6112afac None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:48.305595 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 448/896] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Nov 2 03:22:48 2022] GET /identity/v3/projects/f2ca43aa6a1042f08c4359d0cffed7b3 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:22:48.311327 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9de355f-b7e2-4365-ae79-e36b625a7540 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:48.323144 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:48.406979 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9de355f-b7e2-4365-ae79-e36b625a7540 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:48.439345 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ab8acd-3bdf-49ef-a3d8-891ecf026398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:48.440009 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab8acd-3bdf-49ef-a3d8-891ecf026398 None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:48.440291 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab8acd-3bdf-49ef-a3d8-891ecf026398 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:48.440564 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab8acd-3bdf-49ef-a3d8-891ecf026398 None admin] PATH_INFO: `/v3/projects/f2ca43aa6a1042f08c4359d0cffed7b3/users/40c9ce5c302e4190b9ad14581f349f40/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:48.442436 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ab8acd-3bdf-49ef-a3d8-891ecf026398 None admin] RBAC: Authorizing `identity:create_grant(project_id=f2ca43aa6a1042f08c4359d0cffed7b3, user_id=40c9ce5c302e4190b9ad14581f349f40, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:48.443838 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ab8acd-3bdf-49ef-a3d8-891ecf026398 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:48.455900 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 449/897] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Nov 2 03:22:48 2022] PUT /identity/v3/projects/f2ca43aa6a1042f08c4359d0cffed7b3/users/40c9ce5c302e4190b9ad14581f349f40/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:22:50.550946 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e170bc14-bef1-4e03-acd5-f678a34a1a54 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:50.551440 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e170bc14-bef1-4e03-acd5-f678a34a1a54 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:50.551539 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e170bc14-bef1-4e03-acd5-f678a34a1a54 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:50.552916 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 449/898] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:22:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:22:50.558041 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e744fc-1095-423d-82d5-5cacad9c38ac None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:50.558487 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e744fc-1095-423d-82d5-5cacad9c38ac None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:50.558594 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e744fc-1095-423d-82d5-5cacad9c38ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:50.576288 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-20e744fc-1095-423d-82d5-5cacad9c38ac None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:22:50.577457 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-20e744fc-1095-423d-82d5-5cacad9c38ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:22:50.637522 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:50.642404 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 450/899] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:22:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Nov 02 03:22:50.907816 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e170bc14-bef1-4e03-acd5-f678a34a1a54 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:50.992313 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:51.008281 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e170bc14-bef1-4e03-acd5-f678a34a1a54 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:51.040488 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929bbaee-f8de-439f-a203-3ba68e215d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:51.041099 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929bbaee-f8de-439f-a203-3ba68e215d83 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:51.041377 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929bbaee-f8de-439f-a203-3ba68e215d83 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:51.041649 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929bbaee-f8de-439f-a203-3ba68e215d83 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:51.052266 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929bbaee-f8de-439f-a203-3ba68e215d83 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:22:51.057090 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-929bbaee-f8de-439f-a203-3ba68e215d83 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 03:22:51.057090 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:22:51.057090 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:22:51.057090 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:22:51.057090 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:22:51.057090 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:22:51.057090 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:22:51.057090 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:22:51.057090 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:22:51.057090 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:22:51.057090 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:22:51.057090 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:22:51.057090 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:22:51.057090 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:22:51.057090 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:22:51.057090 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:22:51.058214 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:22:51.058214 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:22:51.058214 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:22:51.058214 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:22:51.058214 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:22:51.058214 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:22:51.058214 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:22:51.058214 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:22:51.058214 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:22:51.058214 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:22:51.058214 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:22:51.058214 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:22:51.058214 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:22:51.058214 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:22:51.058214 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:22:51.058214 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:22:51.059104 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:22:51.059104 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:22:51.059104 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:22:51.059104 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:22:51.059104 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:22:51.059104 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:22:51.059104 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:22:51.059104 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:22:51.059104 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:22:51.059104 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:22:51.059104 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 03:22:51.059104 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:22:51.059104 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 450/900] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Nov 2 03:22:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:22:51.064042 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e744fc-1095-423d-82d5-5cacad9c38ac None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:51.157813 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e744fc-1095-423d-82d5-5cacad9c38ac None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:51.189877 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cc48ea-0f3a-4e6e-97d2-b18ac229969a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:51.190411 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc48ea-0f3a-4e6e-97d2-b18ac229969a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:51.190684 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc48ea-0f3a-4e6e-97d2-b18ac229969a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:51.190952 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cc48ea-0f3a-4e6e-97d2-b18ac229969a None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:51.191595 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cc48ea-0f3a-4e6e-97d2-b18ac229969a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:22:51.191922 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cc48ea-0f3a-4e6e-97d2-b18ac229969a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:51.193288 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cc48ea-0f3a-4e6e-97d2-b18ac229969a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:51.200876 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 451/901] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Nov 2 03:22:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:22:51.206469 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929bbaee-f8de-439f-a203-3ba68e215d83 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:51.302769 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929bbaee-f8de-439f-a203-3ba68e215d83 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:51.335060 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b73dca6-572d-4986-98a6-ef600f541ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:51.335664 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b73dca6-572d-4986-98a6-ef600f541ecb None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:51.335938 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b73dca6-572d-4986-98a6-ef600f541ecb None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:51.336246 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b73dca6-572d-4986-98a6-ef600f541ecb None admin] PATH_INFO: `/v3/users/40c9ce5c302e4190b9ad14581f349f40` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:51.337341 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b73dca6-572d-4986-98a6-ef600f541ecb None admin] RBAC: Authorizing `identity:get_user(user_id=40c9ce5c302e4190b9ad14581f349f40)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:51.338744 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b73dca6-572d-4986-98a6-ef600f541ecb None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:51.340299 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 451/902] 10.222.0.26 () {56 vars in 1177 bytes} [Wed Nov 2 03:22:51 2022] GET /identity/v3/users/40c9ce5c302e4190b9ad14581f349f40 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:22:51.346095 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cc48ea-0f3a-4e6e-97d2-b18ac229969a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:51.404922 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:51.442751 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cc48ea-0f3a-4e6e-97d2-b18ac229969a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:51.474837 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec28d3a-35cf-4b65-a308-e9cb9863cb8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:51.475423 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec28d3a-35cf-4b65-a308-e9cb9863cb8e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:51.475699 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec28d3a-35cf-4b65-a308-e9cb9863cb8e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:51.475987 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec28d3a-35cf-4b65-a308-e9cb9863cb8e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:51.541089 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec28d3a-35cf-4b65-a308-e9cb9863cb8e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:51.542533 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec28d3a-35cf-4b65-a308-e9cb9863cb8e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:51.599787 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:51.638053 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 452/903] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Nov 2 03:22:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 03:22:51.643928 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b73dca6-572d-4986-98a6-ef600f541ecb None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:51.660062 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:22:51.741775 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b73dca6-572d-4986-98a6-ef600f541ecb None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:51.774780 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cac55b-e1f3-4d6d-98cb-510bd652d6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:51.775392 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cac55b-e1f3-4d6d-98cb-510bd652d6cc None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:51.775665 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cac55b-e1f3-4d6d-98cb-510bd652d6cc None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:51.775934 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cac55b-e1f3-4d6d-98cb-510bd652d6cc None admin] PATH_INFO: `/v3/projects/f2ca43aa6a1042f08c4359d0cffed7b3` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:51.777057 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cac55b-e1f3-4d6d-98cb-510bd652d6cc None admin] RBAC: Authorizing `identity:get_project(project_id=f2ca43aa6a1042f08c4359d0cffed7b3)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:51.778455 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cac55b-e1f3-4d6d-98cb-510bd652d6cc None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:51.780085 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 452/904] 10.222.0.26 () {56 vars in 1186 bytes} [Wed Nov 2 03:22:51 2022] GET /identity/v3/projects/f2ca43aa6a1042f08c4359d0cffed7b3 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:22:51.786570 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec28d3a-35cf-4b65-a308-e9cb9863cb8e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:22:51.881223 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec28d3a-35cf-4b65-a308-e9cb9863cb8e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:22:51.912768 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086765d-d0c2-4fba-b2b9-984c0f2ed049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:22:51.913354 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086765d-d0c2-4fba-b2b9-984c0f2ed049 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:22:51.913625 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086765d-d0c2-4fba-b2b9-984c0f2ed049 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:22:51.913889 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086765d-d0c2-4fba-b2b9-984c0f2ed049 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:22:51.914584 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f086765d-d0c2-4fba-b2b9-984c0f2ed049 None admin] RBAC: Adding query filter params (role.id=addc0db6b4ac4362b07fe21e14f89180, scope.project.id=f2ca43aa6a1042f08c4359d0cffed7b3, user.id=40c9ce5c302e4190b9ad14581f349f40) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:22:51.914881 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f086765d-d0c2-4fba-b2b9-984c0f2ed049 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:22:51.916301 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f086765d-d0c2-4fba-b2b9-984c0f2ed049 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:22:51.921911 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 453/905] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Nov 2 03:22:51 2022] GET /identity/v3/role_assignments?user.id=40c9ce5c302e4190b9ad14581f349f40&scope.project.id=f2ca43aa6a1042f08c4359d0cffed7b3&role.id=addc0db6b4ac4362b07fe21e14f89180 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:23:01.771014 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=59 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:23:01.919389 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=87 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:23:46.855661 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae734f-712a-4d2c-9da5-d07403230f05 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:23:46.855661 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae734f-712a-4d2c-9da5-d07403230f05 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:23:46.855661 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae734f-712a-4d2c-9da5-d07403230f05 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:23:46.855661 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 453/906] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:23:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:23:46.861465 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4668fff-29c4-4c7f-bb2c-4d31c3ec40f2 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:23:46.866511 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4668fff-29c4-4c7f-bb2c-4d31c3ec40f2 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:23:46.866801 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4668fff-29c4-4c7f-bb2c-4d31c3ec40f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:23:46.889406 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d4668fff-29c4-4c7f-bb2c-4d31c3ec40f2 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:23:46.890687 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d4668fff-29c4-4c7f-bb2c-4d31c3ec40f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:23:46.969932 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 454/907] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:23:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 03:23:47.267412 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ae734f-712a-4d2c-9da5-d07403230f05 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:23:47.398876 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ae734f-712a-4d2c-9da5-d07403230f05 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:23:47.440396 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61b00ab-81d4-40d4-ae0d-1235415d6817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:23:47.441061 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61b00ab-81d4-40d4-ae0d-1235415d6817 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:23:47.441343 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61b00ab-81d4-40d4-ae0d-1235415d6817 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:23:47.441615 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61b00ab-81d4-40d4-ae0d-1235415d6817 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:23:47.453787 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61b00ab-81d4-40d4-ae0d-1235415d6817 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:23:47.455231 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61b00ab-81d4-40d4-ae0d-1235415d6817 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:23:47.471894 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-a61b00ab-81d4-40d4-ae0d-1235415d6817 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:23:47.471894 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:23:47.471894 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:23:47.471894 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:23:47.471894 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:23:47.471894 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:23:47.471894 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:23:47.471894 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:23:47.471894 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:23:47.471894 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:23:47.471894 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:23:47.471894 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:23:47.471894 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 03:23:47.471894 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 03:23:47.471894 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 03:23:47.471894 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 03:23:47.473763 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:23:47.473763 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:23:47.473763 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:23:47.473763 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:23:47.473763 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:23:47.473763 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:23:47.473763 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:23:47.473763 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:23:47.473763 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:23:47.473763 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:23:47.473763 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:23:47.473763 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:23:47.473763 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:23:47.473763 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:23:47.473763 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:23:47.473763 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:23:47.473763 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 03:23:47.475028 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 03:23:47.475028 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:23:47.475028 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:23:47.475028 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 454/908] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:23:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:23:47.479732 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4668fff-29c4-4c7f-bb2c-4d31c3ec40f2 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:23:47.602791 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4668fff-29c4-4c7f-bb2c-4d31c3ec40f2 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:23:47.633647 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:23:47.639780 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4bd757-94ac-4f8b-a76e-05902f0646d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:23:47.640711 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4bd757-94ac-4f8b-a76e-05902f0646d8 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:23:47.641219 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4bd757-94ac-4f8b-a76e-05902f0646d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:23:47.641709 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4bd757-94ac-4f8b-a76e-05902f0646d8 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:23:47.645620 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4bd757-94ac-4f8b-a76e-05902f0646d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:23:47.645620 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4bd757-94ac-4f8b-a76e-05902f0646d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:23:47.646549 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4bd757-94ac-4f8b-a76e-05902f0646d8 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:23:47.663760 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 455/909] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Nov 2 03:23:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:23:47.665119 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61b00ab-81d4-40d4-ae0d-1235415d6817 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:23:47.726983 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:23:47.784701 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61b00ab-81d4-40d4-ae0d-1235415d6817 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:23:47.833399 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe64201-acc0-4c85-9e27-6d70d5c96340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:23:47.834334 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe64201-acc0-4c85-9e27-6d70d5c96340 None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:23:47.834838 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe64201-acc0-4c85-9e27-6d70d5c96340 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:23:47.835356 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe64201-acc0-4c85-9e27-6d70d5c96340 None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:23:47.836363 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe64201-acc0-4c85-9e27-6d70d5c96340 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:23:47.838330 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe64201-acc0-4c85-9e27-6d70d5c96340 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:23:47.865299 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 455/910] 10.222.0.26 () {60 vars in 1132 bytes} [Wed Nov 2 03:23:47 2022] POST /identity/v3/users => generated 251 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:23:50.351910 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae0d87-8d43-4e48-ae15-178c192ee03c None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:23:50.352553 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae0d87-8d43-4e48-ae15-178c192ee03c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:23:50.352646 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcae0d87-8d43-4e48-ae15-178c192ee03c None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:23:50.354631 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 456/911] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:23:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:23:50.361445 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a03ca-b1df-4c98-9fcb-70d9855e0c78 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:23:50.362605 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a03ca-b1df-4c98-9fcb-70d9855e0c78 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:23:50.362740 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56a03ca-b1df-4c98-9fcb-70d9855e0c78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:23:50.390875 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-a56a03ca-b1df-4c98-9fcb-70d9855e0c78 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:23:50.392367 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a56a03ca-b1df-4c98-9fcb-70d9855e0c78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:23:50.486311 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 456/912] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:23:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 03:23:50.785680 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcae0d87-8d43-4e48-ae15-178c192ee03c None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:23:50.901562 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcae0d87-8d43-4e48-ae15-178c192ee03c None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:23:50.939582 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ae733a-f3ad-4cfb-a9a0-755e3359ae56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:23:50.941715 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ae733a-f3ad-4cfb-a9a0-755e3359ae56 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:23:50.941715 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ae733a-f3ad-4cfb-a9a0-755e3359ae56 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:23:50.941715 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ae733a-f3ad-4cfb-a9a0-755e3359ae56 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:23:50.953181 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ae733a-f3ad-4cfb-a9a0-755e3359ae56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:23:50.958397 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-42ae733a-f3ad-4cfb-a9a0-755e3359ae56 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 03:23:50.958397 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:23:50.958397 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:23:50.958397 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:23:50.958397 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:23:50.958397 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:23:50.958397 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:23:50.958397 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:23:50.958397 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:23:50.958397 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:23:50.958397 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:23:50.958397 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:23:50.958397 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:23:50.958397 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:23:50.958397 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:23:50.958397 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:23:50.959586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:23:50.959586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:23:50.959586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:23:50.959586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:23:50.959586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:23:50.959586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:23:50.959586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:23:50.959586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:23:50.959586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:23:50.959586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:23:50.959586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:23:50.959586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:23:50.959586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:23:50.959586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:23:50.959586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:23:50.959586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:23:50.960639 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:23:50.960639 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:23:50.960639 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:23:50.960639 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:23:50.960639 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:23:50.960639 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:23:50.960639 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:23:50.960639 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:23:50.960639 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:23:50.960639 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:23:50.960639 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 03:23:50.960639 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:23:50.960639 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 457/913] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Nov 2 03:23:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:23:50.965809 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56a03ca-b1df-4c98-9fcb-70d9855e0c78 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:23:51.016567 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:23:51.084506 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56a03ca-b1df-4c98-9fcb-70d9855e0c78 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:23:51.122345 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104bb467-7c2e-42d9-b2cd-ab5dfd194ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:23:51.122925 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104bb467-7c2e-42d9-b2cd-ab5dfd194ce4 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:23:51.123203 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104bb467-7c2e-42d9-b2cd-ab5dfd194ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:23:51.123467 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104bb467-7c2e-42d9-b2cd-ab5dfd194ce4 None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:23:51.124197 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104bb467-7c2e-42d9-b2cd-ab5dfd194ce4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:23:51.124491 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104bb467-7c2e-42d9-b2cd-ab5dfd194ce4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:23:51.125901 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104bb467-7c2e-42d9-b2cd-ab5dfd194ce4 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:23:51.135139 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 457/914] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Nov 2 03:23:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:23:51.142127 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ae733a-f3ad-4cfb-a9a0-755e3359ae56 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:23:51.209916 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:23:51.273906 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ae733a-f3ad-4cfb-a9a0-755e3359ae56 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:23:51.315616 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94515515-f47e-486f-a679-e1562d0feebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:23:51.316596 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94515515-f47e-486f-a679-e1562d0feebb None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:23:51.317088 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94515515-f47e-486f-a679-e1562d0feebb None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:23:51.317357 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94515515-f47e-486f-a679-e1562d0feebb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:23:51.329751 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94515515-f47e-486f-a679-e1562d0feebb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:23:51.332027 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94515515-f47e-486f-a679-e1562d0feebb None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:23:51.347041 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-94515515-f47e-486f-a679-e1562d0feebb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:23:51.347041 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:23:51.347041 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:23:51.347041 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:23:51.347041 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:23:51.347041 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:23:51.347041 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:23:51.347041 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:23:51.347041 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:23:51.347041 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:23:51.347041 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:23:51.347041 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:23:51.347041 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 03:23:51.347041 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 03:23:51.347041 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 03:23:51.347041 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 03:23:51.348238 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:23:51.348238 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:23:51.348238 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:23:51.348238 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:23:51.348238 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:23:51.348238 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:23:51.348238 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:23:51.348238 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:23:51.348238 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:23:51.348238 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:23:51.348238 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:23:51.348238 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:23:51.348238 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:23:51.348238 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:23:51.348238 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:23:51.348238 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:23:51.348238 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 03:23:51.349140 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 03:23:51.349140 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:23:51.349140 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:23:51.349140 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 458/915] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:23:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:23:51.354234 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104bb467-7c2e-42d9-b2cd-ab5dfd194ce4 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:23:51.482458 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:23:51.486520 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104bb467-7c2e-42d9-b2cd-ab5dfd194ce4 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:23:51.527712 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7deabf-73b3-4adf-b906-29a5adf0bc6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:23:51.528568 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7deabf-73b3-4adf-b906-29a5adf0bc6f None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:23:51.528845 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7deabf-73b3-4adf-b906-29a5adf0bc6f None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:23:51.529142 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7deabf-73b3-4adf-b906-29a5adf0bc6f None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:23:51.529885 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7deabf-73b3-4adf-b906-29a5adf0bc6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:23:51.530230 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7deabf-73b3-4adf-b906-29a5adf0bc6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:23:51.531972 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7deabf-73b3-4adf-b906-29a5adf0bc6f None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:23:51.550575 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 458/916] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Nov 2 03:23:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:23:51.556240 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94515515-f47e-486f-a679-e1562d0feebb None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:23:51.668328 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94515515-f47e-486f-a679-e1562d0feebb None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:23:51.668328 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:23:51.702215 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aab90fc-4d6f-447f-bfe5-ede27839e10b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:23:51.703407 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab90fc-4d6f-447f-bfe5-ede27839e10b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:23:51.703407 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab90fc-4d6f-447f-bfe5-ede27839e10b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:23:51.703407 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab90fc-4d6f-447f-bfe5-ede27839e10b None admin] PATH_INFO: `/v3/users/placement` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:23:51.711341 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aab90fc-4d6f-447f-bfe5-ede27839e10b None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:23:51.712830 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aab90fc-4d6f-447f-bfe5-ede27839e10b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:23:51.724954 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-8aab90fc-4d6f-447f-bfe5-ede27839e10b None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Nov 02 03:23:51.724954 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:23:51.724954 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:23:51.724954 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:23:51.724954 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:23:51.724954 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:23:51.724954 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:23:51.724954 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:23:51.724954 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:23:51.724954 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:23:51.724954 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:23:51.724954 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:23:51.724954 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 03:23:51.724954 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 03:23:51.724954 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 03:23:51.724954 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 03:23:51.726008 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:23:51.726008 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:23:51.726008 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 03:23:51.726008 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 03:23:51.726008 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 03:23:51.726008 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 03:23:51.726008 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:23:51.726008 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:23:51.726008 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:23:51.726008 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:23:51.726008 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:23:51.726008 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:23:51.726008 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:23:51.726008 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:23:51.726008 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:23:51.726008 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:23:51.726008 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:23:51.726008 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:23:51.726996 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:23:51.726996 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:23:51.726996 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 03:23:51.726996 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 03:23:51.726996 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 03:23:51.726996 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 03:23:51.726996 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 03:23:51.726996 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 03:23:51.726996 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Nov 02 03:23:51.726996 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:23:51.726996 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 459/917] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:23:51 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:23:51.731940 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7deabf-73b3-4adf-b906-29a5adf0bc6f None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:23:51.841467 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7deabf-73b3-4adf-b906-29a5adf0bc6f None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:23:51.878420 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d975ac7-052c-438e-bd1d-8a92695712a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:23:51.879020 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d975ac7-052c-438e-bd1d-8a92695712a7 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:23:51.879292 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d975ac7-052c-438e-bd1d-8a92695712a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:23:51.879595 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d975ac7-052c-438e-bd1d-8a92695712a7 None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:23:51.880310 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d975ac7-052c-438e-bd1d-8a92695712a7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:23:51.880612 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d975ac7-052c-438e-bd1d-8a92695712a7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:23:51.882020 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d975ac7-052c-438e-bd1d-8a92695712a7 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:23:51.904510 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 459/918] 10.222.0.26 () {56 vars in 1143 bytes} [Wed Nov 2 03:23:51 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:23:51.905751 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aab90fc-4d6f-447f-bfe5-ede27839e10b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:23:52.027480 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aab90fc-4d6f-447f-bfe5-ede27839e10b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:23:52.061476 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff51e20-282d-4e16-be12-0aafc8c7470d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:23:52.062033 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff51e20-282d-4e16-be12-0aafc8c7470d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:23:52.062308 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff51e20-282d-4e16-be12-0aafc8c7470d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:23:52.062577 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff51e20-282d-4e16-be12-0aafc8c7470d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:23:52.073770 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff51e20-282d-4e16-be12-0aafc8c7470d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:23:52.075258 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff51e20-282d-4e16-be12-0aafc8c7470d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:23:52.089653 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-aff51e20-282d-4e16-be12-0aafc8c7470d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:23:52.089653 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:23:52.089653 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:23:52.089653 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:23:52.089653 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:23:52.089653 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:23:52.089653 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:23:52.089653 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:23:52.089653 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:23:52.089653 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:23:52.089653 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:23:52.089653 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:23:52.089653 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 03:23:52.089653 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 03:23:52.089653 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 03:23:52.089653 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 03:23:52.090817 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:23:52.090817 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:23:52.090817 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:23:52.090817 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:23:52.090817 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:23:52.090817 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:23:52.090817 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:23:52.090817 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:23:52.090817 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:23:52.090817 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:23:52.090817 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:23:52.090817 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:23:52.090817 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:23:52.090817 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:23:52.090817 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:23:52.090817 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:23:52.090817 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 03:23:52.091741 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 03:23:52.091741 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:23:52.091741 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:23:52.091741 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 460/919] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:23:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:23:52.096979 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d975ac7-052c-438e-bd1d-8a92695712a7 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:23:52.117774 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:23:52.206740 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d975ac7-052c-438e-bd1d-8a92695712a7 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:23:52.248370 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb24edd-1168-4ce4-8db1-64c207a28169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:23:52.249290 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb24edd-1168-4ce4-8db1-64c207a28169 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:23:52.249826 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb24edd-1168-4ce4-8db1-64c207a28169 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:23:52.250339 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb24edd-1168-4ce4-8db1-64c207a28169 None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:23:52.251296 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb24edd-1168-4ce4-8db1-64c207a28169 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:23:52.251747 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb24edd-1168-4ce4-8db1-64c207a28169 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:23:52.254410 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb24edd-1168-4ce4-8db1-64c207a28169 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:23:52.268231 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 460/920] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Nov 2 03:23:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:23:52.273763 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff51e20-282d-4e16-be12-0aafc8c7470d None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:23:52.292371 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:23:52.393628 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff51e20-282d-4e16-be12-0aafc8c7470d None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:23:52.434290 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5c43f9-ef3d-45ba-82b7-cf83b2967cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:23:52.435092 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5c43f9-ef3d-45ba-82b7-cf83b2967cde None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:23:52.435530 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5c43f9-ef3d-45ba-82b7-cf83b2967cde None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:23:52.435882 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5c43f9-ef3d-45ba-82b7-cf83b2967cde None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:23:52.444643 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5c43f9-ef3d-45ba-82b7-cf83b2967cde None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:23:52.446102 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5c43f9-ef3d-45ba-82b7-cf83b2967cde None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:23:52.458351 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-2b5c43f9-ef3d-45ba-82b7-cf83b2967cde None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 03:23:52.458351 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:23:52.458351 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:23:52.458351 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:23:52.458351 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:23:52.458351 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:23:52.458351 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:23:52.458351 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:23:52.458351 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:23:52.458351 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:23:52.458351 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:23:52.458351 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:23:52.458351 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 03:23:52.458351 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 03:23:52.458351 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 03:23:52.459475 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 03:23:52.459475 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:23:52.459475 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:23:52.459475 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:23:52.459475 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:23:52.459475 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:23:52.459475 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:23:52.459475 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:23:52.459475 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:23:52.459475 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:23:52.459475 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:23:52.459475 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:23:52.459475 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:23:52.459475 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:23:52.459475 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:23:52.459475 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:23:52.459475 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:23:52.460545 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 03:23:52.460545 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 03:23:52.460545 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 03:23:52.460545 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 03:23:52.460545 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 03:23:52.460545 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 03:23:52.460545 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 03:23:52.460545 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:23:52.460545 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 461/921] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Nov 2 03:23:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 03:23:52.465280 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb24edd-1168-4ce4-8db1-64c207a28169 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:23:52.559527 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:23:52.583907 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb24edd-1168-4ce4-8db1-64c207a28169 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:23:52.623460 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ff18ac-401b-4419-9bb0-46f23326b01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:23:52.624116 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ff18ac-401b-4419-9bb0-46f23326b01b None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:23:52.624395 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ff18ac-401b-4419-9bb0-46f23326b01b None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:23:52.624664 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ff18ac-401b-4419-9bb0-46f23326b01b None admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:23:52.625363 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ff18ac-401b-4419-9bb0-46f23326b01b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:23:52.625662 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ff18ac-401b-4419-9bb0-46f23326b01b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:23:52.627108 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ff18ac-401b-4419-9bb0-46f23326b01b None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:23:52.640419 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 461/922] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Nov 2 03:23:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:23:52.647045 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5c43f9-ef3d-45ba-82b7-cf83b2967cde None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:23:52.728953 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:23:52.756374 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5c43f9-ef3d-45ba-82b7-cf83b2967cde None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:23:52.794662 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a01fdc2-786d-457e-9233-877d8aafc31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:23:52.795386 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a01fdc2-786d-457e-9233-877d8aafc31f None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:23:52.795673 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a01fdc2-786d-457e-9233-877d8aafc31f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:23:52.795980 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a01fdc2-786d-457e-9233-877d8aafc31f None admin] PATH_INFO: `/v3/projects/d0b5e7017ac64ae6ab62c1a05aa54b72/users/9b4b9bd741574ddabadd4b7651ebc885/roles/e2a148d6fd534007bbb44d586d4306d1` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:23:52.812692 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a01fdc2-786d-457e-9233-877d8aafc31f None admin] RBAC: Authorizing `identity:create_grant(project_id=d0b5e7017ac64ae6ab62c1a05aa54b72, user_id=9b4b9bd741574ddabadd4b7651ebc885, role_id=e2a148d6fd534007bbb44d586d4306d1)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:23:52.814293 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a01fdc2-786d-457e-9233-877d8aafc31f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:23:52.828994 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 462/923] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Nov 2 03:23:52 2022] PUT /identity/v3/projects/d0b5e7017ac64ae6ab62c1a05aa54b72/users/9b4b9bd741574ddabadd4b7651ebc885/roles/e2a148d6fd534007bbb44d586d4306d1 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:23:55.042014 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393ebea2-ef63-4c81-8071-8e6691995e68 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:23:55.042586 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393ebea2-ef63-4c81-8071-8e6691995e68 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:23:55.042586 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393ebea2-ef63-4c81-8071-8e6691995e68 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:23:55.043885 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 462/924] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:23:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:23:55.049071 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d77169-84df-4460-be7d-202b59e3c0c2 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:23:55.049502 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d77169-84df-4460-be7d-202b59e3c0c2 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:23:55.049622 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d77169-84df-4460-be7d-202b59e3c0c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:23:55.067880 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-b3d77169-84df-4460-be7d-202b59e3c0c2 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:23:55.069098 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b3d77169-84df-4460-be7d-202b59e3c0c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:23:55.138977 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 463/925] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:23:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 03:23:55.416784 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393ebea2-ef63-4c81-8071-8e6691995e68 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:23:55.525596 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393ebea2-ef63-4c81-8071-8e6691995e68 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:23:55.564227 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8004dd32-56cc-493b-9e44-4b9a17742028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:23:55.565587 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8004dd32-56cc-493b-9e44-4b9a17742028 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:23:55.566026 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8004dd32-56cc-493b-9e44-4b9a17742028 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:23:55.566444 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8004dd32-56cc-493b-9e44-4b9a17742028 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:23:55.574050 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:23:55.578527 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8004dd32-56cc-493b-9e44-4b9a17742028 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:23:55.584995 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-8004dd32-56cc-493b-9e44-4b9a17742028 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Nov 02 03:23:55.584995 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:23:55.584995 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:23:55.584995 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:23:55.584995 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:23:55.584995 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:23:55.584995 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:23:55.584995 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:23:55.584995 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:23:55.584995 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:23:55.584995 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:23:55.584995 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:23:55.584995 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:23:55.584995 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:23:55.584995 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:23:55.584995 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:23:55.586103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:23:55.586103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:23:55.586103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:23:55.586103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:23:55.586103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:23:55.586103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:23:55.586103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:23:55.586103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:23:55.586103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:23:55.586103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:23:55.586103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:23:55.586103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:23:55.586103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:23:55.586103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:23:55.586103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:23:55.586103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:23:55.586974 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:23:55.586974 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:23:55.586974 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:23:55.586974 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:23:55.586974 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:23:55.586974 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:23:55.586974 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:23:55.586974 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:23:55.586974 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:23:55.586974 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:23:55.586974 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Nov 02 03:23:55.586974 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:23:55.587637 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 463/926] 10.222.0.26 () {56 vars in 1102 bytes} [Wed Nov 2 03:23:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:23:55.593958 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d77169-84df-4460-be7d-202b59e3c0c2 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:23:55.655737 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:23:55.703423 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d77169-84df-4460-be7d-202b59e3c0c2 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:23:55.740421 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe99d3e-1149-44b3-96c8-61f700b9e76c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:23:55.741034 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe99d3e-1149-44b3-96c8-61f700b9e76c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:23:55.741312 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe99d3e-1149-44b3-96c8-61f700b9e76c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:23:55.741581 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe99d3e-1149-44b3-96c8-61f700b9e76c None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:23:55.742248 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe99d3e-1149-44b3-96c8-61f700b9e76c None admin] RBAC: Adding query filter params (name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:23:55.742546 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe99d3e-1149-44b3-96c8-61f700b9e76c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:23:55.743906 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe99d3e-1149-44b3-96c8-61f700b9e76c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:23:55.751971 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 464/927] 10.222.0.26 () {56 vars in 1103 bytes} [Wed Nov 2 03:23:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:23:55.758393 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8004dd32-56cc-493b-9e44-4b9a17742028 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:23:55.875358 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8004dd32-56cc-493b-9e44-4b9a17742028 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:23:55.914705 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a68795-1363-4246-800f-55ccd4d843e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:23:55.915334 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a68795-1363-4246-800f-55ccd4d843e5 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:23:55.915614 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a68795-1363-4246-800f-55ccd4d843e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:23:55.915889 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a68795-1363-4246-800f-55ccd4d843e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:23:55.927147 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a68795-1363-4246-800f-55ccd4d843e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:23:55.928843 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a68795-1363-4246-800f-55ccd4d843e5 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:23:55.944948 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-82a68795-1363-4246-800f-55ccd4d843e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:23:55.944948 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:23:55.944948 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:23:55.944948 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:23:55.944948 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:23:55.944948 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:23:55.944948 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:23:55.944948 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:23:55.944948 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:23:55.944948 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:23:55.944948 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:23:55.944948 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:23:55.944948 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 03:23:55.944948 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 03:23:55.944948 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 03:23:55.944948 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 03:23:55.946056 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:23:55.946056 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:23:55.946056 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:23:55.946056 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:23:55.946056 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:23:55.946056 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:23:55.946056 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:23:55.946056 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:23:55.946056 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:23:55.946056 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:23:55.946056 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:23:55.946056 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:23:55.946056 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:23:55.946056 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:23:55.946056 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:23:55.946056 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:23:55.946056 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 03:23:55.946971 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 03:23:55.946971 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:23:55.946971 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:23:55.946971 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 464/928] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:23:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 03:23:55.952122 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe99d3e-1149-44b3-96c8-61f700b9e76c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:23:56.058117 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe99d3e-1149-44b3-96c8-61f700b9e76c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:23:56.075049 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:23:56.094718 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c27af82-db85-4658-a254-d7e7c88a2b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:23:56.095320 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c27af82-db85-4658-a254-d7e7c88a2b3d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:23:56.095594 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c27af82-db85-4658-a254-d7e7c88a2b3d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:23:56.095879 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c27af82-db85-4658-a254-d7e7c88a2b3d None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:23:56.096537 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c27af82-db85-4658-a254-d7e7c88a2b3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:23:56.096837 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c27af82-db85-4658-a254-d7e7c88a2b3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:23:56.098186 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c27af82-db85-4658-a254-d7e7c88a2b3d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:23:56.110370 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 465/929] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Nov 2 03:23:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:23:56.116283 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a68795-1363-4246-800f-55ccd4d843e5 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:23:56.171811 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:23:56.228425 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a68795-1363-4246-800f-55ccd4d843e5 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:23:56.269089 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7071410c-ee5a-4c78-8f14-a11639d3169d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:23:56.269724 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7071410c-ee5a-4c78-8f14-a11639d3169d None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:23:56.270008 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7071410c-ee5a-4c78-8f14-a11639d3169d None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:23:56.270285 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7071410c-ee5a-4c78-8f14-a11639d3169d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:23:56.278032 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7071410c-ee5a-4c78-8f14-a11639d3169d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:23:56.279577 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7071410c-ee5a-4c78-8f14-a11639d3169d None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:23:56.291793 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-7071410c-ee5a-4c78-8f14-a11639d3169d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Nov 02 03:23:56.291793 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:23:56.291793 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:23:56.291793 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:23:56.291793 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:23:56.291793 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:23:56.291793 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:23:56.291793 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:23:56.291793 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:23:56.291793 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:23:56.291793 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:23:56.291793 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:23:56.291793 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 03:23:56.291793 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 03:23:56.291793 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 03:23:56.291793 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 03:23:56.292964 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:23:56.292964 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:23:56.292964 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 03:23:56.292964 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 03:23:56.292964 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 03:23:56.292964 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 03:23:56.292964 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:23:56.292964 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:23:56.292964 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:23:56.292964 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:23:56.292964 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:23:56.292964 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:23:56.292964 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:23:56.292964 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:23:56.292964 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:23:56.292964 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:23:56.292964 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:23:56.292964 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:23:56.294224 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:23:56.294224 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:23:56.294224 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 03:23:56.294224 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 03:23:56.294224 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 03:23:56.294224 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 03:23:56.294224 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 03:23:56.294224 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 03:23:56.294224 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Nov 02 03:23:56.294224 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:23:56.294224 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 465/930] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:23:56 2022] GET /identity/v3/users/placement => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:23:56.298929 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c27af82-db85-4658-a254-d7e7c88a2b3d None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:23:56.411885 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c27af82-db85-4658-a254-d7e7c88a2b3d None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:23:56.447535 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6907339d-8d07-4d00-91cc-506eb1bcd50d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:23:56.448176 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907339d-8d07-4d00-91cc-506eb1bcd50d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:23:56.448456 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907339d-8d07-4d00-91cc-506eb1bcd50d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:23:56.448730 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907339d-8d07-4d00-91cc-506eb1bcd50d None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:23:56.449448 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6907339d-8d07-4d00-91cc-506eb1bcd50d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:23:56.449752 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6907339d-8d07-4d00-91cc-506eb1bcd50d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:23:56.451293 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6907339d-8d07-4d00-91cc-506eb1bcd50d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:23:56.471055 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 466/931] 10.222.0.26 () {56 vars in 1143 bytes} [Wed Nov 2 03:23:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:23:56.477265 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7071410c-ee5a-4c78-8f14-a11639d3169d None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:23:56.589131 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7071410c-ee5a-4c78-8f14-a11639d3169d None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:23:56.600233 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:23:56.625914 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36af11fb-c26f-4db5-b21b-8ee171510e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:23:56.626525 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af11fb-c26f-4db5-b21b-8ee171510e80 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:23:56.626807 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af11fb-c26f-4db5-b21b-8ee171510e80 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:23:56.627216 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af11fb-c26f-4db5-b21b-8ee171510e80 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:23:56.701715 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36af11fb-c26f-4db5-b21b-8ee171510e80 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:23:56.703106 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36af11fb-c26f-4db5-b21b-8ee171510e80 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:23:56.809652 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 466/932] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Nov 2 03:23:56 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 03:23:56.815605 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6907339d-8d07-4d00-91cc-506eb1bcd50d None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:23:56.852020 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:23:56.925455 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6907339d-8d07-4d00-91cc-506eb1bcd50d None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:23:56.961226 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c867968f-6ea6-455e-81f2-ebf5be5666b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:23:56.962119 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c867968f-6ea6-455e-81f2-ebf5be5666b6 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:23:56.962449 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c867968f-6ea6-455e-81f2-ebf5be5666b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:23:56.962759 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c867968f-6ea6-455e-81f2-ebf5be5666b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:23:56.975074 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c867968f-6ea6-455e-81f2-ebf5be5666b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:23:56.976856 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c867968f-6ea6-455e-81f2-ebf5be5666b6 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:23:56.993458 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-c867968f-6ea6-455e-81f2-ebf5be5666b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:23:56.993458 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:23:56.993458 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:23:56.993458 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:23:56.993458 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:23:56.993458 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:23:56.993458 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:23:56.993458 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:23:56.993458 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:23:56.993458 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:23:56.993458 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:23:56.993458 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:23:56.993458 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 03:23:56.993458 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 03:23:56.993458 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 03:23:56.993458 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 03:23:56.994632 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:23:56.994632 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:23:56.994632 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:23:56.994632 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:23:56.994632 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:23:56.994632 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:23:56.994632 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:23:56.994632 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:23:56.994632 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:23:56.994632 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:23:56.994632 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:23:56.994632 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:23:56.994632 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:23:56.994632 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:23:56.994632 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:23:56.994632 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:23:56.994632 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 03:23:56.995542 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 03:23:56.995542 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:23:56.995542 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:23:56.995542 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 467/933] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:23:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:23:57.000834 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36af11fb-c26f-4db5-b21b-8ee171510e80 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:23:57.014770 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:23:57.110745 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36af11fb-c26f-4db5-b21b-8ee171510e80 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:23:57.145832 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f75f24-e193-4f74-9a8a-701c45e6696a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:23:57.146447 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f75f24-e193-4f74-9a8a-701c45e6696a None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:23:57.146702 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f75f24-e193-4f74-9a8a-701c45e6696a None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:23:57.147014 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f75f24-e193-4f74-9a8a-701c45e6696a None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:23:57.147782 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f75f24-e193-4f74-9a8a-701c45e6696a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:23:57.148122 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f75f24-e193-4f74-9a8a-701c45e6696a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:23:57.149484 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f75f24-e193-4f74-9a8a-701c45e6696a None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:23:57.161780 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 467/934] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Nov 2 03:23:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:23:57.167573 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c867968f-6ea6-455e-81f2-ebf5be5666b6 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:23:57.261172 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:23:57.277924 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c867968f-6ea6-455e-81f2-ebf5be5666b6 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:23:57.314575 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b82cdc-ef62-4679-893e-97eb0cf4bf21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:23:57.315237 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b82cdc-ef62-4679-893e-97eb0cf4bf21 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:23:57.315517 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b82cdc-ef62-4679-893e-97eb0cf4bf21 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:23:57.315795 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b82cdc-ef62-4679-893e-97eb0cf4bf21 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:23:57.322456 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b82cdc-ef62-4679-893e-97eb0cf4bf21 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:23:57.323937 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b82cdc-ef62-4679-893e-97eb0cf4bf21 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:23:57.335723 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-14b82cdc-ef62-4679-893e-97eb0cf4bf21 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 03:23:57.335723 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:23:57.335723 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:23:57.335723 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:23:57.335723 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:23:57.335723 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:23:57.335723 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:23:57.335723 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:23:57.335723 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:23:57.335723 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:23:57.335723 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:23:57.335723 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:23:57.335723 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 03:23:57.335723 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 03:23:57.335723 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 03:23:57.336677 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 03:23:57.336677 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:23:57.336677 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:23:57.336677 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:23:57.336677 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:23:57.336677 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:23:57.336677 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:23:57.336677 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:23:57.336677 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:23:57.336677 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:23:57.336677 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:23:57.336677 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:23:57.336677 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:23:57.336677 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:23:57.336677 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:23:57.336677 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:23:57.336677 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:23:57.337608 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 03:23:57.337608 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 03:23:57.337608 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 03:23:57.337608 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 03:23:57.337608 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 03:23:57.337608 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 03:23:57.337608 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 03:23:57.337608 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:23:57.337608 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 468/935] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Nov 2 03:23:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:23:57.343463 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f75f24-e193-4f74-9a8a-701c45e6696a None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:23:57.417391 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:23:57.452796 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f75f24-e193-4f74-9a8a-701c45e6696a None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:23:57.490036 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bad8225-e3c2-41b9-9a70-ca928cd3ea67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:23:57.490695 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bad8225-e3c2-41b9-9a70-ca928cd3ea67 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:23:57.491027 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bad8225-e3c2-41b9-9a70-ca928cd3ea67 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:23:57.491343 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bad8225-e3c2-41b9-9a70-ca928cd3ea67 None admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:23:57.492114 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bad8225-e3c2-41b9-9a70-ca928cd3ea67 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:23:57.492462 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bad8225-e3c2-41b9-9a70-ca928cd3ea67 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:23:57.493900 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bad8225-e3c2-41b9-9a70-ca928cd3ea67 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:23:57.507094 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 468/936] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Nov 2 03:23:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:23:57.513457 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b82cdc-ef62-4679-893e-97eb0cf4bf21 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:23:57.620123 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b82cdc-ef62-4679-893e-97eb0cf4bf21 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:23:57.656664 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91a79e0-8e53-497f-9611-4218b3e48f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:23:57.657280 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a79e0-8e53-497f-9611-4218b3e48f9c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:23:57.657587 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a79e0-8e53-497f-9611-4218b3e48f9c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:23:57.657865 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91a79e0-8e53-497f-9611-4218b3e48f9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:23:57.658648 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91a79e0-8e53-497f-9611-4218b3e48f9c None admin] RBAC: Adding query filter params (role.id=e2a148d6fd534007bbb44d586d4306d1, scope.project.id=d0b5e7017ac64ae6ab62c1a05aa54b72, user.id=9b4b9bd741574ddabadd4b7651ebc885) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:23:57.658908 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91a79e0-8e53-497f-9611-4218b3e48f9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:23:57.660359 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91a79e0-8e53-497f-9611-4218b3e48f9c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:23:57.666160 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 469/937] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Nov 2 03:23:57 2022] GET /identity/v3/role_assignments?user.id=9b4b9bd741574ddabadd4b7651ebc885&scope.project.id=d0b5e7017ac64ae6ab62c1a05aa54b72&role.id=e2a148d6fd534007bbb44d586d4306d1 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:23:59.911343 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad1166-cf7e-486b-b3e2-0145b57c443a None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:23:59.911889 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad1166-cf7e-486b-b3e2-0145b57c443a None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:23:59.912108 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cad1166-cf7e-486b-b3e2-0145b57c443a None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:23:59.913456 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 469/938] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:23:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 03:23:59.919081 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681714bc-ccc7-4753-b002-791c9d83e8b1 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:23:59.920003 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681714bc-ccc7-4753-b002-791c9d83e8b1 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:23:59.920003 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681714bc-ccc7-4753-b002-791c9d83e8b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:23:59.940111 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-681714bc-ccc7-4753-b002-791c9d83e8b1 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:23:59.941599 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-681714bc-ccc7-4753-b002-791c9d83e8b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:23:59.945310 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:24:00.012874 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 470/939] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:23:59 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 03:24:00.302758 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cad1166-cf7e-486b-b3e2-0145b57c443a None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:24:00.414470 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cad1166-cf7e-486b-b3e2-0145b57c443a None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:24:00.446244 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:24:00.452804 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99d1500-faf7-4dac-bd91-62a4dbfcfd82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:24:00.453435 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99d1500-faf7-4dac-bd91-62a4dbfcfd82 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:24:00.453716 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99d1500-faf7-4dac-bd91-62a4dbfcfd82 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:24:00.453990 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99d1500-faf7-4dac-bd91-62a4dbfcfd82 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:24:00.464520 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99d1500-faf7-4dac-bd91-62a4dbfcfd82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:24:00.469374 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-f99d1500-faf7-4dac-bd91-62a4dbfcfd82 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 03:24:00.469374 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:24:00.469374 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:24:00.469374 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:24:00.469374 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:24:00.469374 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:24:00.469374 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:24:00.469374 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:24:00.469374 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:24:00.469374 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:24:00.469374 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:24:00.469374 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:24:00.469374 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:24:00.469374 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:24:00.469374 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:24:00.469374 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:24:00.470466 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:24:00.470466 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:24:00.470466 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:24:00.470466 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:24:00.470466 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:24:00.470466 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:24:00.470466 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:24:00.470466 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:24:00.470466 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:24:00.470466 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:24:00.470466 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:24:00.470466 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:24:00.470466 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:24:00.470466 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:24:00.470466 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:24:00.470466 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:24:00.472087 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:24:00.472087 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:24:00.472087 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:24:00.472087 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:24:00.472087 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:24:00.472087 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:24:00.472087 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:24:00.472087 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:24:00.472087 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:24:00.472087 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:24:00.472087 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 03:24:00.472087 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:24:00.472087 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 470/940] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Nov 2 03:24:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 03:24:00.477086 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681714bc-ccc7-4753-b002-791c9d83e8b1 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:24:00.588616 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681714bc-ccc7-4753-b002-791c9d83e8b1 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:24:00.626424 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66294208-ff89-4864-b3ac-442b8b16943d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:24:00.627595 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66294208-ff89-4864-b3ac-442b8b16943d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:24:00.627998 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66294208-ff89-4864-b3ac-442b8b16943d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:24:00.628233 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66294208-ff89-4864-b3ac-442b8b16943d None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:24:00.628918 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66294208-ff89-4864-b3ac-442b8b16943d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:24:00.629233 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66294208-ff89-4864-b3ac-442b8b16943d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:24:00.630824 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66294208-ff89-4864-b3ac-442b8b16943d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:24:00.639528 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 471/941] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Nov 2 03:24:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:24:00.646036 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99d1500-faf7-4dac-bd91-62a4dbfcfd82 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:24:00.760443 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99d1500-faf7-4dac-bd91-62a4dbfcfd82 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:24:00.797071 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e98240-2ceb-4361-8bae-05156165ebd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:24:00.797866 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e98240-2ceb-4361-8bae-05156165ebd5 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:24:00.798184 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e98240-2ceb-4361-8bae-05156165ebd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:24:00.798538 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e98240-2ceb-4361-8bae-05156165ebd5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:24:00.809551 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e98240-2ceb-4361-8bae-05156165ebd5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:24:00.811191 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e98240-2ceb-4361-8bae-05156165ebd5 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:24:00.828141 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-e5e98240-2ceb-4361-8bae-05156165ebd5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:24:00.828141 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:24:00.828141 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:24:00.828141 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:24:00.828141 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:24:00.828141 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:24:00.828141 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:24:00.828141 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:24:00.828141 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:24:00.828141 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:24:00.828141 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:24:00.828141 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:24:00.828141 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 03:24:00.828141 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 03:24:00.828141 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 03:24:00.828141 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 03:24:00.829350 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:24:00.829350 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:24:00.829350 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:24:00.829350 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:24:00.829350 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:24:00.829350 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:24:00.829350 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:24:00.829350 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:24:00.829350 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:24:00.829350 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:24:00.829350 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:24:00.829350 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:24:00.829350 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:24:00.829350 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:24:00.829350 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:24:00.829350 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:24:00.829350 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 03:24:00.830343 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 03:24:00.830343 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:24:00.830343 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:24:00.830343 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 471/942] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:24:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:24:00.835432 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66294208-ff89-4864-b3ac-442b8b16943d None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:24:00.838545 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:24:00.949404 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66294208-ff89-4864-b3ac-442b8b16943d None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:24:00.986345 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e816c3d-164b-457b-86ea-c2b540c7dd6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:24:00.986959 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e816c3d-164b-457b-86ea-c2b540c7dd6e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:24:00.987506 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e816c3d-164b-457b-86ea-c2b540c7dd6e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:24:00.987854 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e816c3d-164b-457b-86ea-c2b540c7dd6e None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:24:00.988540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e816c3d-164b-457b-86ea-c2b540c7dd6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:24:00.988841 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e816c3d-164b-457b-86ea-c2b540c7dd6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:24:00.990210 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e816c3d-164b-457b-86ea-c2b540c7dd6e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:24:01.002711 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 472/943] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Nov 2 03:24:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:24:01.008745 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e98240-2ceb-4361-8bae-05156165ebd5 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:24:01.055237 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:24:01.118137 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e98240-2ceb-4361-8bae-05156165ebd5 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:24:01.155365 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9938b24e-83b4-4273-abea-1d81cc1b29d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:24:01.156122 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9938b24e-83b4-4273-abea-1d81cc1b29d5 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:24:01.156468 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9938b24e-83b4-4273-abea-1d81cc1b29d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:24:01.156803 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9938b24e-83b4-4273-abea-1d81cc1b29d5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:24:01.164332 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9938b24e-83b4-4273-abea-1d81cc1b29d5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:24:01.165740 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9938b24e-83b4-4273-abea-1d81cc1b29d5 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:24:01.177147 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-9938b24e-83b4-4273-abea-1d81cc1b29d5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Nov 02 03:24:01.177147 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:24:01.177147 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:24:01.177147 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:24:01.177147 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:24:01.177147 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:24:01.177147 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:24:01.177147 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:24:01.177147 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:24:01.177147 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:24:01.177147 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:24:01.177147 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:24:01.177147 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 03:24:01.177147 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 03:24:01.177147 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 03:24:01.177147 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 03:24:01.178265 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:24:01.178265 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:24:01.178265 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 03:24:01.178265 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 03:24:01.178265 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 03:24:01.178265 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 03:24:01.178265 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:24:01.178265 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:24:01.178265 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:24:01.178265 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:24:01.178265 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:24:01.178265 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:24:01.178265 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:24:01.178265 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:24:01.178265 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:24:01.178265 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:24:01.178265 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:24:01.178265 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:24:01.179231 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:24:01.179231 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:24:01.179231 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 03:24:01.179231 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 03:24:01.179231 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 03:24:01.179231 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 03:24:01.179231 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 03:24:01.179231 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 03:24:01.179231 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Nov 02 03:24:01.179231 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:24:01.179231 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 472/944] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:24:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:24:01.184014 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e816c3d-164b-457b-86ea-c2b540c7dd6e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:24:01.256242 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:24:01.301315 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e816c3d-164b-457b-86ea-c2b540c7dd6e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:24:01.340193 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721cdc20-ceed-4ae2-86bc-1d32227e38ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:24:01.340829 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721cdc20-ceed-4ae2-86bc-1d32227e38ac None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:24:01.341171 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721cdc20-ceed-4ae2-86bc-1d32227e38ac None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:24:01.341453 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721cdc20-ceed-4ae2-86bc-1d32227e38ac None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:24:01.342272 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721cdc20-ceed-4ae2-86bc-1d32227e38ac None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:24:01.342577 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721cdc20-ceed-4ae2-86bc-1d32227e38ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:24:01.344065 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721cdc20-ceed-4ae2-86bc-1d32227e38ac None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:24:01.363989 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 473/945] 10.222.0.26 () {56 vars in 1143 bytes} [Wed Nov 2 03:24:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:24:01.370444 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9938b24e-83b4-4273-abea-1d81cc1b29d5 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:24:01.481257 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9938b24e-83b4-4273-abea-1d81cc1b29d5 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:24:01.482871 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:24:01.521401 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6248d3-4224-4c45-ba22-ac1eb795b8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:24:01.521956 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6248d3-4224-4c45-ba22-ac1eb795b8d3 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:24:01.522226 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6248d3-4224-4c45-ba22-ac1eb795b8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:24:01.522534 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6248d3-4224-4c45-ba22-ac1eb795b8d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:24:01.539384 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6248d3-4224-4c45-ba22-ac1eb795b8d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:24:01.541005 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6248d3-4224-4c45-ba22-ac1eb795b8d3 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:24:01.557112 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-8a6248d3-4224-4c45-ba22-ac1eb795b8d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:24:01.557112 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:24:01.557112 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:24:01.557112 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:24:01.557112 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:24:01.557112 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:24:01.557112 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:24:01.557112 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:24:01.557112 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:24:01.557112 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:24:01.557112 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:24:01.557112 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:24:01.557112 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 03:24:01.557112 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 03:24:01.557112 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 03:24:01.557112 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 03:24:01.558432 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:24:01.558432 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:24:01.558432 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:24:01.558432 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:24:01.558432 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:24:01.558432 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:24:01.558432 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:24:01.558432 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:24:01.558432 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:24:01.558432 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:24:01.558432 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:24:01.558432 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:24:01.558432 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:24:01.558432 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:24:01.558432 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:24:01.558432 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:24:01.558432 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 03:24:01.559354 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 03:24:01.559354 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:24:01.559354 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:24:01.559354 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 473/946] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:24:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 03:24:01.564171 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721cdc20-ceed-4ae2-86bc-1d32227e38ac None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:24:01.674833 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721cdc20-ceed-4ae2-86bc-1d32227e38ac None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:24:01.697958 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:24:01.714220 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8879630a-de9b-444b-bdf7-6a3cf216ae6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:24:01.714858 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8879630a-de9b-444b-bdf7-6a3cf216ae6b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:24:01.715132 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8879630a-de9b-444b-bdf7-6a3cf216ae6b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:24:01.715402 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8879630a-de9b-444b-bdf7-6a3cf216ae6b None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:24:01.716080 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8879630a-de9b-444b-bdf7-6a3cf216ae6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:24:01.716379 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8879630a-de9b-444b-bdf7-6a3cf216ae6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:24:01.717730 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8879630a-de9b-444b-bdf7-6a3cf216ae6b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:24:01.730743 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 474/947] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Nov 2 03:24:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:24:01.737850 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6248d3-4224-4c45-ba22-ac1eb795b8d3 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:24:01.854017 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6248d3-4224-4c45-ba22-ac1eb795b8d3 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:24:01.890558 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d76811-efd5-4a2c-90ca-ceaa39dbf7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:24:01.891172 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d76811-efd5-4a2c-90ca-ceaa39dbf7f6 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:24:01.891430 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d76811-efd5-4a2c-90ca-ceaa39dbf7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:24:01.891697 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d76811-efd5-4a2c-90ca-ceaa39dbf7f6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:24:01.898683 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d76811-efd5-4a2c-90ca-ceaa39dbf7f6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:24:01.900377 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d76811-efd5-4a2c-90ca-ceaa39dbf7f6 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:24:01.911492 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-a9d76811-efd5-4a2c-90ca-ceaa39dbf7f6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 03:24:01.911492 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:24:01.911492 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:24:01.911492 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:24:01.911492 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:24:01.911492 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:24:01.911492 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:24:01.911492 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:24:01.911492 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:24:01.911492 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:24:01.911492 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:24:01.911492 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:24:01.911492 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 03:24:01.911492 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 03:24:01.911492 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 03:24:01.912648 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 03:24:01.912648 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:24:01.912648 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:24:01.912648 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:24:01.912648 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:24:01.912648 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:24:01.912648 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:24:01.912648 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:24:01.912648 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:24:01.912648 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:24:01.912648 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:24:01.912648 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:24:01.912648 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:24:01.912648 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:24:01.912648 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:24:01.912648 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:24:01.912648 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:24:01.913934 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 03:24:01.913934 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 03:24:01.913934 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 03:24:01.913934 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 03:24:01.913934 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 03:24:01.913934 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 03:24:01.913934 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 03:24:01.913934 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:24:01.913934 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 474/948] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Nov 2 03:24:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 03:24:01.918487 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8879630a-de9b-444b-bdf7-6a3cf216ae6b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:24:02.020507 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8879630a-de9b-444b-bdf7-6a3cf216ae6b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:24:02.057211 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b2fbd6-e359-4f95-b333-19f6e55280d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:24:02.057832 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2fbd6-e359-4f95-b333-19f6e55280d0 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:24:02.058113 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2fbd6-e359-4f95-b333-19f6e55280d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:24:02.058384 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2fbd6-e359-4f95-b333-19f6e55280d0 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:24:02.059184 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b2fbd6-e359-4f95-b333-19f6e55280d0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:24:02.059535 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b2fbd6-e359-4f95-b333-19f6e55280d0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:24:02.061233 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b2fbd6-e359-4f95-b333-19f6e55280d0 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:24:02.075119 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 475/949] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Nov 2 03:24:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:24:02.081052 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d76811-efd5-4a2c-90ca-ceaa39dbf7f6 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:24:02.099897 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:24:02.194888 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d76811-efd5-4a2c-90ca-ceaa39dbf7f6 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:24:02.230972 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5787734f-e43f-4bbf-a633-3f3c4bb4dc14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:24:02.231753 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5787734f-e43f-4bbf-a633-3f3c4bb4dc14 None admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:24:02.232188 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5787734f-e43f-4bbf-a633-3f3c4bb4dc14 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:24:02.232526 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5787734f-e43f-4bbf-a633-3f3c4bb4dc14 None admin] PATH_INFO: `/v3/projects/d0b5e7017ac64ae6ab62c1a05aa54b72/users/9b4b9bd741574ddabadd4b7651ebc885/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:24:02.235008 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5787734f-e43f-4bbf-a633-3f3c4bb4dc14 None admin] RBAC: Authorizing `identity:create_grant(project_id=d0b5e7017ac64ae6ab62c1a05aa54b72, user_id=9b4b9bd741574ddabadd4b7651ebc885, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:24:02.236934 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5787734f-e43f-4bbf-a633-3f3c4bb4dc14 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:24:02.254543 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 475/950] 10.222.0.26 () {58 vars in 1439 bytes} [Wed Nov 2 03:24:02 2022] PUT /identity/v3/projects/d0b5e7017ac64ae6ab62c1a05aa54b72/users/9b4b9bd741574ddabadd4b7651ebc885/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:24:04.577830 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c2798-1962-4fe7-ad3a-00dc923a97eb None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:24:04.578376 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c2798-1962-4fe7-ad3a-00dc923a97eb None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:24:04.578376 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c2798-1962-4fe7-ad3a-00dc923a97eb None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:24:04.579931 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 476/951] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:24:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:24:04.585313 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03dbb77-75c0-4f56-818b-272d02543394 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:24:04.585769 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03dbb77-75c0-4f56-818b-272d02543394 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:24:04.585862 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03dbb77-75c0-4f56-818b-272d02543394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:24:04.605185 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-a03dbb77-75c0-4f56-818b-272d02543394 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:24:04.606615 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a03dbb77-75c0-4f56-818b-272d02543394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:24:04.677627 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 476/952] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:24:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 03:24:04.944801 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272c2798-1962-4fe7-ad3a-00dc923a97eb None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:24:04.992265 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:24:05.059714 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272c2798-1962-4fe7-ad3a-00dc923a97eb None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:24:05.098951 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4627b2a-ea84-4c8a-a733-79ac790887af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:24:05.099702 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4627b2a-ea84-4c8a-a733-79ac790887af None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:24:05.100038 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4627b2a-ea84-4c8a-a733-79ac790887af None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:24:05.100330 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4627b2a-ea84-4c8a-a733-79ac790887af None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:24:05.112246 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4627b2a-ea84-4c8a-a733-79ac790887af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Nov 02 03:24:05.117142 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-f4627b2a-ea84-4c8a-a733-79ac790887af None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 03:24:05.117142 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:24:05.117142 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:24:05.117142 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:24:05.117142 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:24:05.117142 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:24:05.117142 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:24:05.117142 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:24:05.117142 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:24:05.117142 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:24:05.117142 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:24:05.117142 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:24:05.117142 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Nov 02 03:24:05.117142 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Nov 02 03:24:05.117142 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Nov 02 03:24:05.117142 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Nov 02 03:24:05.118175 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Nov 02 03:24:05.118175 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Nov 02 03:24:05.118175 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Nov 02 03:24:05.118175 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Nov 02 03:24:05.118175 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:24:05.118175 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:24:05.118175 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:24:05.118175 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:24:05.118175 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:24:05.118175 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:24:05.118175 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:24:05.118175 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:24:05.118175 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:24:05.118175 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:24:05.118175 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:24:05.118175 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:24:05.119040 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:24:05.119040 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:24:05.119040 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:24:05.119040 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:24:05.119040 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Nov 02 03:24:05.119040 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Nov 02 03:24:05.119040 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Nov 02 03:24:05.119040 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Nov 02 03:24:05.119040 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Nov 02 03:24:05.119040 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Nov 02 03:24:05.119040 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Nov 02 03:24:05.119040 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:24:05.119040 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 477/953] 10.222.0.26 () {56 vars in 1096 bytes} [Wed Nov 2 03:24:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:24:05.125134 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03dbb77-75c0-4f56-818b-272d02543394 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:24:05.128342 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:24:05.228709 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03dbb77-75c0-4f56-818b-272d02543394 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:24:05.280112 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f3b710-038b-42fb-853c-919ed018d7ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:24:05.280924 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f3b710-038b-42fb-853c-919ed018d7ce None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:24:05.281240 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f3b710-038b-42fb-853c-919ed018d7ce None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:24:05.281623 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f3b710-038b-42fb-853c-919ed018d7ce None admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:24:05.282497 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f3b710-038b-42fb-853c-919ed018d7ce None admin] RBAC: Adding query filter params (name=admin) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:24:05.282874 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f3b710-038b-42fb-853c-919ed018d7ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:24:05.284802 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f3b710-038b-42fb-853c-919ed018d7ce None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:24:05.295661 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 477/954] 10.222.0.26 () {56 vars in 1099 bytes} [Wed Nov 2 03:24:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:24:05.302442 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4627b2a-ea84-4c8a-a733-79ac790887af None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:24:05.407055 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:24:05.410086 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4627b2a-ea84-4c8a-a733-79ac790887af None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:24:05.444574 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb328d-c725-4226-912d-53c59706d601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:24:05.445205 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb328d-c725-4226-912d-53c59706d601 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:24:05.445481 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb328d-c725-4226-912d-53c59706d601 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:24:05.445753 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bb328d-c725-4226-912d-53c59706d601 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:24:05.455686 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bb328d-c725-4226-912d-53c59706d601 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:24:05.457151 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bb328d-c725-4226-912d-53c59706d601 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:24:05.471567 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-28bb328d-c725-4226-912d-53c59706d601 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:24:05.471567 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:24:05.471567 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:24:05.471567 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:24:05.471567 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:24:05.471567 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:24:05.471567 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:24:05.471567 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:24:05.471567 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:24:05.471567 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:24:05.471567 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:24:05.471567 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:24:05.471567 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 03:24:05.471567 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 03:24:05.471567 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 03:24:05.471567 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 03:24:05.472800 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:24:05.472800 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:24:05.472800 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:24:05.472800 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:24:05.472800 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:24:05.472800 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:24:05.472800 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:24:05.472800 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:24:05.472800 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:24:05.472800 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:24:05.472800 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:24:05.472800 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:24:05.472800 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:24:05.472800 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:24:05.472800 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:24:05.472800 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:24:05.472800 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 03:24:05.474090 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 03:24:05.474090 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:24:05.474090 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:24:05.474090 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 478/955] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:24:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:24:05.478801 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f3b710-038b-42fb-853c-919ed018d7ce None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:24:05.555596 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:24:05.599662 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f3b710-038b-42fb-853c-919ed018d7ce None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:24:05.634005 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb9b4ee-4087-470d-bac0-99eb65fa72bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:24:05.634614 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb9b4ee-4087-470d-bac0-99eb65fa72bb None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:24:05.634900 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb9b4ee-4087-470d-bac0-99eb65fa72bb None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:24:05.635173 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb9b4ee-4087-470d-bac0-99eb65fa72bb None admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:24:05.635813 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb9b4ee-4087-470d-bac0-99eb65fa72bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:24:05.636152 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb9b4ee-4087-470d-bac0-99eb65fa72bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:24:05.637510 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb9b4ee-4087-470d-bac0-99eb65fa72bb None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:24:05.649585 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 478/956] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Nov 2 03:24:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:24:05.655458 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb328d-c725-4226-912d-53c59706d601 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:24:05.766403 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bb328d-c725-4226-912d-53c59706d601 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:24:05.802832 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb3bf7a-5439-4514-9ae9-65c0d340534d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:24:05.803599 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb3bf7a-5439-4514-9ae9-65c0d340534d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:24:05.804022 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb3bf7a-5439-4514-9ae9-65c0d340534d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:24:05.804375 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb3bf7a-5439-4514-9ae9-65c0d340534d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:24:05.811997 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb3bf7a-5439-4514-9ae9-65c0d340534d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:24:05.813548 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb3bf7a-5439-4514-9ae9-65c0d340534d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:24:05.824965 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-bfb3bf7a-5439-4514-9ae9-65c0d340534d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Nov 02 03:24:05.824965 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:24:05.824965 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:24:05.824965 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:24:05.824965 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:24:05.824965 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:24:05.824965 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:24:05.824965 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:24:05.824965 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:24:05.824965 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:24:05.824965 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:24:05.824965 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:24:05.824965 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Nov 02 03:24:05.824965 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Nov 02 03:24:05.824965 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Nov 02 03:24:05.824965 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Nov 02 03:24:05.826148 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:24:05.826148 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:24:05.826148 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Nov 02 03:24:05.826148 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 03:24:05.826148 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Nov 02 03:24:05.826148 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Nov 02 03:24:05.826148 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:24:05.826148 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:24:05.826148 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:24:05.826148 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:24:05.826148 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:24:05.826148 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:24:05.826148 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:24:05.826148 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:24:05.826148 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:24:05.826148 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:24:05.826148 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:24:05.826148 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:24:05.827354 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:24:05.827354 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:24:05.827354 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Nov 02 03:24:05.827354 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Nov 02 03:24:05.827354 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Nov 02 03:24:05.827354 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Nov 02 03:24:05.827354 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Nov 02 03:24:05.827354 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Nov 02 03:24:05.827354 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Nov 02 03:24:05.827354 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:24:05.831082 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 479/957] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:24:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:24:05.832080 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb9b4ee-4087-470d-bac0-99eb65fa72bb None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:24:05.945005 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb9b4ee-4087-470d-bac0-99eb65fa72bb None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:24:05.974031 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:24:05.983879 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a05035e-b23f-4d75-a9ee-acf9db326e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:24:05.984544 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05035e-b23f-4d75-a9ee-acf9db326e56 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:24:05.984838 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05035e-b23f-4d75-a9ee-acf9db326e56 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:24:05.985072 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05035e-b23f-4d75-a9ee-acf9db326e56 None admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:24:05.985940 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a05035e-b23f-4d75-a9ee-acf9db326e56 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:24:05.986884 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a05035e-b23f-4d75-a9ee-acf9db326e56 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:24:05.988734 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a05035e-b23f-4d75-a9ee-acf9db326e56 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:24:06.006876 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 479/958] 10.222.0.26 () {56 vars in 1143 bytes} [Wed Nov 2 03:24:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:24:06.012874 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb3bf7a-5439-4514-9ae9-65c0d340534d None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:24:06.029024 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:24:06.120230 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb3bf7a-5439-4514-9ae9-65c0d340534d None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:24:06.159086 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3aa5f7-f6da-460f-b60b-6ed300d5f568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:24:06.159709 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3aa5f7-f6da-460f-b60b-6ed300d5f568 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:24:06.160009 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3aa5f7-f6da-460f-b60b-6ed300d5f568 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:24:06.160289 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d3aa5f7-f6da-460f-b60b-6ed300d5f568 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:24:06.238748 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3aa5f7-f6da-460f-b60b-6ed300d5f568 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:24:06.240595 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d3aa5f7-f6da-460f-b60b-6ed300d5f568 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:24:06.256596 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:24:06.352965 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 480/959] 10.222.0.26 () {58 vars in 1282 bytes} [Wed Nov 2 03:24:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Nov 02 03:24:06.358485 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a05035e-b23f-4d75-a9ee-acf9db326e56 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:24:06.470328 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a05035e-b23f-4d75-a9ee-acf9db326e56 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:24:06.507682 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab91f62-da4f-469d-bfe9-d2bae66b26e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:24:06.508361 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab91f62-da4f-469d-bfe9-d2bae66b26e1 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:24:06.508640 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab91f62-da4f-469d-bfe9-d2bae66b26e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:24:06.508910 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab91f62-da4f-469d-bfe9-d2bae66b26e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:24:06.520230 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab91f62-da4f-469d-bfe9-d2bae66b26e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:24:06.521719 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab91f62-da4f-469d-bfe9-d2bae66b26e1 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:24:06.538125 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-0ab91f62-da4f-469d-bfe9-d2bae66b26e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:24:06.538125 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:24:06.538125 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:24:06.538125 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:24:06.538125 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:24:06.538125 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:24:06.538125 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:24:06.538125 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:24:06.538125 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:24:06.538125 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:24:06.538125 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:24:06.538125 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:24:06.538125 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Nov 02 03:24:06.538125 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Nov 02 03:24:06.538125 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Nov 02 03:24:06.538125 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Nov 02 03:24:06.539355 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:24:06.539355 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:24:06.539355 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:24:06.539355 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:24:06.539355 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:24:06.539355 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:24:06.539355 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:24:06.539355 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:24:06.539355 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:24:06.539355 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:24:06.539355 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:24:06.539355 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:24:06.539355 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:24:06.539355 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:24:06.539355 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:24:06.539355 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:24:06.539355 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Nov 02 03:24:06.540340 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Nov 02 03:24:06.540340 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Nov 02 03:24:06.540340 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:24:06.540340 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 480/960] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:24:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:24:06.545396 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3aa5f7-f6da-460f-b60b-6ed300d5f568 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:24:06.647329 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d3aa5f7-f6da-460f-b60b-6ed300d5f568 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:24:06.661985 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:24:06.685970 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b515e91-0909-4eeb-b98f-a62e09767f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:24:06.686586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b515e91-0909-4eeb-b98f-a62e09767f15 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:24:06.686863 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b515e91-0909-4eeb-b98f-a62e09767f15 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:24:06.687131 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b515e91-0909-4eeb-b98f-a62e09767f15 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:24:06.687782 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b515e91-0909-4eeb-b98f-a62e09767f15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:24:06.688111 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b515e91-0909-4eeb-b98f-a62e09767f15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:24:06.689523 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b515e91-0909-4eeb-b98f-a62e09767f15 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:24:06.701386 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 481/961] 10.222.0.26 () {56 vars in 1109 bytes} [Wed Nov 2 03:24:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:24:06.707245 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab91f62-da4f-469d-bfe9-d2bae66b26e1 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:24:06.744695 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:24:06.819552 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab91f62-da4f-469d-bfe9-d2bae66b26e1 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:24:06.853328 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb1303c-5088-4fb7-997f-33271605bf79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:24:06.853954 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb1303c-5088-4fb7-997f-33271605bf79 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:24:06.854233 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb1303c-5088-4fb7-997f-33271605bf79 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:24:06.854506 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb1303c-5088-4fb7-997f-33271605bf79 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:24:06.861647 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb1303c-5088-4fb7-997f-33271605bf79 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:24:06.863343 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb1303c-5088-4fb7-997f-33271605bf79 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:24:06.874497 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application [None req-4fb1303c-5088-4fb7-997f-33271605bf79 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 03:24:06.874497 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:24:06.874497 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:24:06.874497 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:24:06.874497 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:24:06.874497 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:24:06.874497 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:24:06.874497 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:24:06.874497 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:24:06.874497 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:24:06.874497 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:24:06.874497 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:24:06.874497 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Nov 02 03:24:06.874497 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(project_id) Nov 02 03:24:06.874497 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Nov 02 03:24:06.875554 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Nov 02 03:24:06.875554 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:24:06.875554 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:24:06.875554 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:24:06.875554 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:24:06.875554 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:24:06.875554 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:24:06.875554 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:24:06.875554 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application with Lock( Nov 02 03:24:06.875554 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:24:06.875554 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:24:06.875554 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:24:06.875554 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:24:06.875554 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:24:06.875554 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:24:06.875554 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:24:06.875554 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:24:06.876634 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 03:24:06.876634 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 03:24:06.876634 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 03:24:06.876634 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 03:24:06.876634 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 03:24:06.876634 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 03:24:06.876634 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Nov 02 03:24:06.876634 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: ERROR keystone.server.flask.application Nov 02 03:24:06.876634 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 481/962] 10.222.0.26 () {56 vars in 1111 bytes} [Wed Nov 2 03:24:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Nov 02 03:24:06.881577 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b515e91-0909-4eeb-b98f-a62e09767f15 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:24:06.982611 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b515e91-0909-4eeb-b98f-a62e09767f15 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:24:07.018478 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcabbbe6-e001-4c1d-bbd7-9843ff01c439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:24:07.019085 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcabbbe6-e001-4c1d-bbd7-9843ff01c439 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:24:07.019360 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcabbbe6-e001-4c1d-bbd7-9843ff01c439 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:24:07.019625 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcabbbe6-e001-4c1d-bbd7-9843ff01c439 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:24:07.020335 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcabbbe6-e001-4c1d-bbd7-9843ff01c439 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:24:07.020639 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcabbbe6-e001-4c1d-bbd7-9843ff01c439 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:24:07.022041 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcabbbe6-e001-4c1d-bbd7-9843ff01c439 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:24:07.034628 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 482/963] 10.222.0.26 () {56 vars in 1148 bytes} [Wed Nov 2 03:24:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:24:07.040724 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb1303c-5088-4fb7-997f-33271605bf79 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:24:07.134057 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:24:07.140063 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb1303c-5088-4fb7-997f-33271605bf79 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:24:07.172894 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4c34a3-5201-4ff6-ad82-eceb4adf85b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:24:07.173486 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c34a3-5201-4ff6-ad82-eceb4adf85b6 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:24:07.173795 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c34a3-5201-4ff6-ad82-eceb4adf85b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:24:07.174066 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c34a3-5201-4ff6-ad82-eceb4adf85b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:24:07.174777 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4c34a3-5201-4ff6-ad82-eceb4adf85b6 None admin] RBAC: Adding query filter params (role.id=addc0db6b4ac4362b07fe21e14f89180, scope.project.id=d0b5e7017ac64ae6ab62c1a05aa54b72, user.id=9b4b9bd741574ddabadd4b7651ebc885) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:24:07.175083 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4c34a3-5201-4ff6-ad82-eceb4adf85b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:24:07.176536 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4c34a3-5201-4ff6-ad82-eceb4adf85b6 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:24:07.182065 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 482/964] 10.222.0.26 () {56 vars in 1374 bytes} [Wed Nov 2 03:24:07 2022] GET /identity/v3/role_assignments?user.id=9b4b9bd741574ddabadd4b7651ebc885&scope.project.id=d0b5e7017ac64ae6ab62c1a05aa54b72&role.id=addc0db6b4ac4362b07fe21e14f89180 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:24:09.303637 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f785c-d56c-4c4f-ae72-ad1ff3e513cf None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:24:09.304219 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f785c-d56c-4c4f-ae72-ad1ff3e513cf None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:24:09.304219 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f785c-d56c-4c4f-ae72-ad1ff3e513cf None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:24:09.305554 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 483/965] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:24:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:24:09.310584 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384c699-bc14-4bd4-a33f-2512311b9ba2 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:24:09.310964 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384c699-bc14-4bd4-a33f-2512311b9ba2 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:24:09.311138 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384c699-bc14-4bd4-a33f-2512311b9ba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:24:09.329388 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-6384c699-bc14-4bd4-a33f-2512311b9ba2 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:24:09.330585 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-6384c699-bc14-4bd4-a33f-2512311b9ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:24:09.394950 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 483/966] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:24:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 03:24:09.659244 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f785c-d56c-4c4f-ae72-ad1ff3e513cf None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:24:09.689570 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:24:09.757447 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f785c-d56c-4c4f-ae72-ad1ff3e513cf None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:24:09.789718 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71819c5-0c4e-4d63-bf0b-b72299de8c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:24:09.790330 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71819c5-0c4e-4d63-bf0b-b72299de8c20 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:24:09.790607 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71819c5-0c4e-4d63-bf0b-b72299de8c20 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:24:09.790874 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71819c5-0c4e-4d63-bf0b-b72299de8c20 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:24:09.791442 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71819c5-0c4e-4d63-bf0b-b72299de8c20 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:24:09.792819 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71819c5-0c4e-4d63-bf0b-b72299de8c20 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:24:09.801842 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-d71819c5-0c4e-4d63-bf0b-b72299de8c20 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 03:24:09.801842 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:24:09.801842 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 03:24:09.801842 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 03:24:09.801842 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 03:24:09.801842 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 03:24:09.801842 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 03:24:09.801842 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 03:24:09.801842 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 03:24:09.801842 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:24:09.801842 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 03:24:09.801842 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:24:09.801842 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:24:09.801842 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:24:09.801842 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:24:09.801842 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:24:09.801842 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:24:09.803038 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:24:09.803038 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:24:09.803038 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:24:09.803038 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:24:09.803038 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:24:09.803038 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:24:09.803038 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 03:24:09.803038 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 03:24:09.803038 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 03:24:09.803038 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 03:24:09.803038 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:24:09.803038 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:24:09.803038 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:24:09.803038 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:24:09.803038 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:24:09.803038 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:24:09.803902 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:24:09.803902 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:24:09.803902 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:24:09.803902 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:24:09.803902 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:24:09.803902 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:24:09.803902 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:24:09.803902 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:24:09.803902 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:24:09.803902 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:24:09.803902 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 03:24:09.803902 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 03:24:09.803902 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 03:24:09.803902 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:24:09.803902 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 484/967] 10.222.0.26 () {56 vars in 1117 bytes} [Wed Nov 2 03:24:09 2022] GET /identity/v3/services/placement => generated 90 bytes in 145 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:24:09.808768 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6384c699-bc14-4bd4-a33f-2512311b9ba2 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:24:09.894726 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:24:09.904181 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6384c699-bc14-4bd4-a33f-2512311b9ba2 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:24:09.936739 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584236da-74ff-479e-808c-53590446350d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:24:09.937277 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584236da-74ff-479e-808c-53590446350d None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:24:09.937556 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584236da-74ff-479e-808c-53590446350d None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:24:09.937832 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584236da-74ff-479e-808c-53590446350d None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:24:09.938450 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584236da-74ff-479e-808c-53590446350d None admin] RBAC: Adding query filter params (name=placement) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:24:09.938747 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584236da-74ff-479e-808c-53590446350d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:24:09.940127 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584236da-74ff-479e-808c-53590446350d None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:24:09.944738 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 484/968] 10.222.0.26 () {56 vars in 1116 bytes} [Wed Nov 2 03:24:09 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:24:09.950265 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71819c5-0c4e-4d63-bf0b-b72299de8c20 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:24:10.042684 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:24:10.048651 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71819c5-0c4e-4d63-bf0b-b72299de8c20 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:24:10.081172 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780c961b-b9f2-4ccd-be92-f920ab33a598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:24:10.081774 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c961b-b9f2-4ccd-be92-f920ab33a598 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:24:10.082054 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c961b-b9f2-4ccd-be92-f920ab33a598 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:24:10.082328 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780c961b-b9f2-4ccd-be92-f920ab33a598 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:24:10.082945 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780c961b-b9f2-4ccd-be92-f920ab33a598 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:24:10.083244 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780c961b-b9f2-4ccd-be92-f920ab33a598 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:24:10.084623 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780c961b-b9f2-4ccd-be92-f920ab33a598 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:24:10.088946 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 485/969] 10.222.0.26 () {56 vars in 1116 bytes} [Wed Nov 2 03:24:09 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:24:12.210664 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df303b-dd9a-4d7a-b53f-cc2458f186c8 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:24:12.211257 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df303b-dd9a-4d7a-b53f-cc2458f186c8 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:24:12.211257 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8df303b-dd9a-4d7a-b53f-cc2458f186c8 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:24:12.212585 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 485/970] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:24:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:24:12.217684 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f515354-226f-4f01-b1b6-dc8ac0e67f3f None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:24:12.218097 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f515354-226f-4f01-b1b6-dc8ac0e67f3f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:24:12.218240 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f515354-226f-4f01-b1b6-dc8ac0e67f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:24:12.236575 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-3f515354-226f-4f01-b1b6-dc8ac0e67f3f None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:24:12.238002 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3f515354-226f-4f01-b1b6-dc8ac0e67f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:24:12.301859 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 486/971] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:24:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 03:24:12.569398 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8df303b-dd9a-4d7a-b53f-cc2458f186c8 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:24:12.667972 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8df303b-dd9a-4d7a-b53f-cc2458f186c8 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:24:12.700494 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9b01cf-8ef8-4273-9d59-b65c6f2b72f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:24:12.701082 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b01cf-8ef8-4273-9d59-b65c6f2b72f6 None admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:24:12.701358 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b01cf-8ef8-4273-9d59-b65c6f2b72f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:24:12.701628 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9b01cf-8ef8-4273-9d59-b65c6f2b72f6 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:24:12.702273 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9b01cf-8ef8-4273-9d59-b65c6f2b72f6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:24:12.703624 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9b01cf-8ef8-4273-9d59-b65c6f2b72f6 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:24:12.717266 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 486/972] 10.222.0.26 () {60 vars in 1141 bytes} [Wed Nov 2 03:24:12 2022] POST /identity/v3/services => generated 246 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:24:14.831117 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503a83cf-1071-4e14-9ccf-33e2ab9b3744 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:24:14.831610 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503a83cf-1071-4e14-9ccf-33e2ab9b3744 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:24:14.831610 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503a83cf-1071-4e14-9ccf-33e2ab9b3744 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:24:14.832959 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 487/973] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:24:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:24:14.838007 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce374767-cf06-4425-80c3-0796555e571f None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:24:14.838440 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce374767-cf06-4425-80c3-0796555e571f None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:24:14.838542 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce374767-cf06-4425-80c3-0796555e571f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:24:14.856884 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-ce374767-cf06-4425-80c3-0796555e571f None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:24:14.858077 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ce374767-cf06-4425-80c3-0796555e571f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:24:14.868110 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:24:14.930859 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 487/974] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:24:14 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 03:24:15.193565 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503a83cf-1071-4e14-9ccf-33e2ab9b3744 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:24:15.294648 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:24:15.305512 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503a83cf-1071-4e14-9ccf-33e2ab9b3744 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:24:15.338176 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33526b17-395a-46e7-83ce-0ee675590fc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:24:15.338798 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33526b17-395a-46e7-83ce-0ee675590fc6 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:24:15.339075 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33526b17-395a-46e7-83ce-0ee675590fc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:24:15.339345 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33526b17-395a-46e7-83ce-0ee675590fc6 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:24:15.339917 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33526b17-395a-46e7-83ce-0ee675590fc6 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:24:15.341291 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33526b17-395a-46e7-83ce-0ee675590fc6 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:24:15.349779 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-33526b17-395a-46e7-83ce-0ee675590fc6 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 03:24:15.349779 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:24:15.349779 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 03:24:15.349779 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 03:24:15.349779 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 03:24:15.349779 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 03:24:15.349779 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 03:24:15.349779 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 03:24:15.349779 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 03:24:15.349779 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:24:15.349779 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 03:24:15.349779 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:24:15.349779 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:24:15.349779 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:24:15.349779 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:24:15.349779 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:24:15.349779 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:24:15.351119 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:24:15.351119 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:24:15.351119 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:24:15.351119 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:24:15.351119 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:24:15.351119 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:24:15.351119 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 03:24:15.351119 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 03:24:15.351119 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 03:24:15.351119 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 03:24:15.351119 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:24:15.351119 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:24:15.351119 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:24:15.351119 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:24:15.351119 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:24:15.351119 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:24:15.352090 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:24:15.352090 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:24:15.352090 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:24:15.352090 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:24:15.352090 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:24:15.352090 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:24:15.352090 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:24:15.352090 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:24:15.352090 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:24:15.352090 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:24:15.352090 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 03:24:15.352090 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 03:24:15.352090 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 03:24:15.352090 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:24:15.352090 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 488/975] 10.222.0.26 () {56 vars in 1117 bytes} [Wed Nov 2 03:24:15 2022] GET /identity/v3/services/placement => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:24:15.356986 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce374767-cf06-4425-80c3-0796555e571f None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:24:15.453367 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce374767-cf06-4425-80c3-0796555e571f None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:24:15.486594 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dafabea-fa6c-48f0-a26c-5a82021dfe39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:24:15.487459 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dafabea-fa6c-48f0-a26c-5a82021dfe39 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:24:15.487739 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dafabea-fa6c-48f0-a26c-5a82021dfe39 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:24:15.488040 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dafabea-fa6c-48f0-a26c-5a82021dfe39 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:24:15.488663 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dafabea-fa6c-48f0-a26c-5a82021dfe39 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:24:15.488961 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dafabea-fa6c-48f0-a26c-5a82021dfe39 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:24:15.490312 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dafabea-fa6c-48f0-a26c-5a82021dfe39 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:24:15.495287 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 488/976] 10.222.0.26 () {56 vars in 1116 bytes} [Wed Nov 2 03:24:15 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:24:15.501267 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33526b17-395a-46e7-83ce-0ee675590fc6 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:24:15.598857 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33526b17-395a-46e7-83ce-0ee675590fc6 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:24:15.631309 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3659a589-4a63-43ff-ae3e-c3d3cf586626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:24:15.631900 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659a589-4a63-43ff-ae3e-c3d3cf586626 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:24:15.632216 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659a589-4a63-43ff-ae3e-c3d3cf586626 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:24:15.632487 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659a589-4a63-43ff-ae3e-c3d3cf586626 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:24:15.633173 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3659a589-4a63-43ff-ae3e-c3d3cf586626 None admin] RBAC: Adding query filter params (interface=public, service_id=f6ef502bd0e541ddbdc62613bef84a69, region_id=RegionOne) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:24:15.633477 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3659a589-4a63-43ff-ae3e-c3d3cf586626 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:24:15.634828 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3659a589-4a63-43ff-ae3e-c3d3cf586626 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:24:15.639666 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 489/977] 10.222.0.26 () {56 vars in 1251 bytes} [Wed Nov 2 03:24:15 2022] GET /identity/v3/endpoints?service_id=f6ef502bd0e541ddbdc62613bef84a69&interface=public®ion_id=RegionOne => generated 194 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:24:15.645286 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dafabea-fa6c-48f0-a26c-5a82021dfe39 None admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:24:15.649009 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:24:15.744236 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dafabea-fa6c-48f0-a26c-5a82021dfe39 None admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:24:15.776684 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18e8d16-60e5-4bb5-b1ed-494e2147471f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:24:15.777277 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e8d16-60e5-4bb5-b1ed-494e2147471f None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:24:15.777552 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e8d16-60e5-4bb5-b1ed-494e2147471f None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:24:15.777823 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18e8d16-60e5-4bb5-b1ed-494e2147471f None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:24:15.778417 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18e8d16-60e5-4bb5-b1ed-494e2147471f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:24:15.779762 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18e8d16-60e5-4bb5-b1ed-494e2147471f None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:24:15.786901 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 489/978] 10.222.0.26 () {56 vars in 1087 bytes} [Wed Nov 2 03:24:15 2022] GET /identity/v3/services => generated 2191 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:24:17.901951 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c12c1e8-5ce5-4a53-bb11-9006d7a083a3 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:24:17.902461 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c12c1e8-5ce5-4a53-bb11-9006d7a083a3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:24:17.902461 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c12c1e8-5ce5-4a53-bb11-9006d7a083a3 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:24:17.903762 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 490/979] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:24:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:24:17.909524 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c40d0-44bf-47a0-9872-8961d033faef None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:24:17.909913 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c40d0-44bf-47a0-9872-8961d033faef None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:24:17.910064 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2c40d0-44bf-47a0-9872-8961d033faef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:24:17.928470 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-4a2c40d0-44bf-47a0-9872-8961d033faef None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:24:17.929647 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-4a2c40d0-44bf-47a0-9872-8961d033faef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:24:17.988661 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:24:17.994253 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 490/980] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:24:17 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Nov 02 03:24:18.266361 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c12c1e8-5ce5-4a53-bb11-9006d7a083a3 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:24:18.290838 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:24:18.378021 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c12c1e8-5ce5-4a53-bb11-9006d7a083a3 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:24:18.414599 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04b16ff-c779-4f1e-9b4d-0e9b85ae0eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:24:18.415227 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b16ff-c779-4f1e-9b4d-0e9b85ae0eee None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:24:18.415506 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b16ff-c779-4f1e-9b4d-0e9b85ae0eee None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:24:18.415777 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04b16ff-c779-4f1e-9b4d-0e9b85ae0eee None admin] PATH_INFO: `/v3/services/placement` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:24:18.416380 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04b16ff-c779-4f1e-9b4d-0e9b85ae0eee None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:24:18.417732 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04b16ff-c779-4f1e-9b4d-0e9b85ae0eee None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:24:18.426345 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-a04b16ff-c779-4f1e-9b4d-0e9b85ae0eee None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 03:24:18.426345 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:24:18.426345 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Nov 02 03:24:18.426345 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Nov 02 03:24:18.426345 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Nov 02 03:24:18.426345 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Nov 02 03:24:18.426345 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Nov 02 03:24:18.426345 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 03:24:18.426345 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 03:24:18.426345 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:24:18.426345 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Nov 02 03:24:18.426345 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:24:18.426345 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 03:24:18.426345 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 03:24:18.426345 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 03:24:18.426345 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 03:24:18.426345 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 03:24:18.427577 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 03:24:18.427577 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 03:24:18.427577 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 03:24:18.427577 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 03:24:18.427577 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 03:24:18.427577 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 03:24:18.427577 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Nov 02 03:24:18.427577 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Nov 02 03:24:18.427577 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Nov 02 03:24:18.427577 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Nov 02 03:24:18.427577 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 03:24:18.427577 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 03:24:18.427577 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 03:24:18.427577 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 03:24:18.427577 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 03:24:18.427577 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 03:24:18.428586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 03:24:18.428586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 03:24:18.428586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 03:24:18.428586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 03:24:18.428586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 03:24:18.428586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 03:24:18.428586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 03:24:18.428586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 03:24:18.428586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 03:24:18.428586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 03:24:18.428586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Nov 02 03:24:18.428586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Nov 02 03:24:18.428586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Nov 02 03:24:18.428586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 03:24:18.428586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 491/981] 10.222.0.26 () {56 vars in 1117 bytes} [Wed Nov 2 03:24:18 2022] GET /identity/v3/services/placement => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Nov 02 03:24:18.434465 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2c40d0-44bf-47a0-9872-8961d033faef None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:24:18.535752 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2c40d0-44bf-47a0-9872-8961d033faef None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:24:18.570827 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428e208a-a4d1-4632-b719-249d2e760bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:24:18.571435 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428e208a-a4d1-4632-b719-249d2e760bf8 None admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:24:18.571713 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428e208a-a4d1-4632-b719-249d2e760bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:24:18.572005 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428e208a-a4d1-4632-b719-249d2e760bf8 None admin] PATH_INFO: `/v3/services` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:24:18.572626 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428e208a-a4d1-4632-b719-249d2e760bf8 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:24:18.572927 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428e208a-a4d1-4632-b719-249d2e760bf8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:24:18.574287 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428e208a-a4d1-4632-b719-249d2e760bf8 None admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:24:18.579298 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 491/982] 10.222.0.26 () {56 vars in 1116 bytes} [Wed Nov 2 03:24:18 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:24:18.586490 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04b16ff-c779-4f1e-9b4d-0e9b85ae0eee None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:24:18.670117 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:24:18.689961 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04b16ff-c779-4f1e-9b4d-0e9b85ae0eee None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:24:18.735594 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d3b800-401c-416c-80ad-4ed1c6782d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:24:18.736492 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d3b800-401c-416c-80ad-4ed1c6782d2a None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:24:18.737135 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d3b800-401c-416c-80ad-4ed1c6782d2a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:24:18.737518 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d3b800-401c-416c-80ad-4ed1c6782d2a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:24:18.738476 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d3b800-401c-416c-80ad-4ed1c6782d2a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:24:18.740496 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d3b800-401c-416c-80ad-4ed1c6782d2a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:24:18.763533 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 492/983] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:24:18 2022] POST /identity/v3/endpoints => generated 331 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:24:28.576347 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=69 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:24:28.758962 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:25:22.393017 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8963e3b-3978-491f-b4f5-7ac1dcb6a614 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:25:22.393491 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8963e3b-3978-491f-b4f5-7ac1dcb6a614 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:25:22.393584 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8963e3b-3978-491f-b4f5-7ac1dcb6a614 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:25:22.400040 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f610676-6d4d-4924-80dc-0842dd07975c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:25:22.400440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f610676-6d4d-4924-80dc-0842dd07975c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:25:22.400601 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f610676-6d4d-4924-80dc-0842dd07975c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:25:22.405800 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 492/984] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:25:22 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:25:22.487755 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-8f610676-6d4d-4924-80dc-0842dd07975c None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:25:22.489013 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8f610676-6d4d-4924-80dc-0842dd07975c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:25:22.617335 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 493/985] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:25:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 03:25:22.919639 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce3668-ae66-47ff-9dd8-d97021ad0fbf None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:25:22.920358 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce3668-ae66-47ff-9dd8-d97021ad0fbf None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:25:22.920891 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ce3668-ae66-47ff-9dd8-d97021ad0fbf None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:25:22.922942 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 493/986] 10.222.0.26 () {54 vars in 965 bytes} [Wed Nov 2 03:25:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:25:22.929843 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb5b37b-b64a-4b0d-8baa-8116471e4a60 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:25:22.930160 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb5b37b-b64a-4b0d-8baa-8116471e4a60 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:25:22.930573 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb5b37b-b64a-4b0d-8baa-8116471e4a60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:25:23.028935 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-2fb5b37b-b64a-4b0d-8baa-8116471e4a60 None None] MFA Rules not processed for user `f07e1b51b00f42b7b74c3689268635cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:25:23.030175 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2fb5b37b-b64a-4b0d-8baa-8116471e4a60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:25:23.053470 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:25:23.117723 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 494/987] 10.222.0.26 () {58 vars in 1062 bytes} [Wed Nov 2 03:25:22 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:25:23.127050 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce3668-ae66-47ff-9dd8-d97021ad0fbf None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:25:23.128841 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-19ce3668-ae66-47ff-9dd8-d97021ad0fbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:25:23.241673 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ce3668-ae66-47ff-9dd8-d97021ad0fbf None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:25:23.319117 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6581b862-7cdf-4b86-aebd-611216209fb3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:25:23.319822 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6581b862-7cdf-4b86-aebd-611216209fb3 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:25:23.320221 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6581b862-7cdf-4b86-aebd-611216209fb3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:25:23.320551 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6581b862-7cdf-4b86-aebd-611216209fb3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:25:23.322267 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-6581b862-7cdf-4b86-aebd-611216209fb3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:25:23.381190 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:25:23.440463 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6581b862-7cdf-4b86-aebd-611216209fb3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:25:23.441911 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6581b862-7cdf-4b86-aebd-611216209fb3 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:25:23.461150 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 494/988] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:25:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:25:25.921235 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30468fda-6f99-47da-971b-83bb9f5ce01d None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:25:25.921751 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30468fda-6f99-47da-971b-83bb9f5ce01d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:25:25.921751 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30468fda-6f99-47da-971b-83bb9f5ce01d None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:25:25.923102 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 495/989] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:25:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:25:25.928459 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df95c701-65c4-4c0a-a7fc-6a0dbb09ffd8 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:25:25.928863 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df95c701-65c4-4c0a-a7fc-6a0dbb09ffd8 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:25:25.929004 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df95c701-65c4-4c0a-a7fc-6a0dbb09ffd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:25:26.000630 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-df95c701-65c4-4c0a-a7fc-6a0dbb09ffd8 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:25:26.001737 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-df95c701-65c4-4c0a-a7fc-6a0dbb09ffd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:25:26.099729 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 495/990] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:25:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:25:26.381957 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30468fda-6f99-47da-971b-83bb9f5ce01d None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:25:26.383584 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-30468fda-6f99-47da-971b-83bb9f5ce01d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:25:26.496606 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30468fda-6f99-47da-971b-83bb9f5ce01d None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:25:26.578914 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:25:26.580989 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d848710-40ba-46f1-83af-4bec54da50a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:25:26.581638 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d848710-40ba-46f1-83af-4bec54da50a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:25:26.581934 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d848710-40ba-46f1-83af-4bec54da50a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:25:26.582468 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d848710-40ba-46f1-83af-4bec54da50a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:25:26.584562 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1d848710-40ba-46f1-83af-4bec54da50a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:25:26.694299 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d848710-40ba-46f1-83af-4bec54da50a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:25:26.695653 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d848710-40ba-46f1-83af-4bec54da50a5 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:25:26.714944 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 496/991] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:25:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:25:29.149874 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab2ff3f-980c-4db2-9a8e-3bba4ddbbaf8 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:25:29.150407 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab2ff3f-980c-4db2-9a8e-3bba4ddbbaf8 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:25:29.150407 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab2ff3f-980c-4db2-9a8e-3bba4ddbbaf8 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:25:29.151776 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 496/992] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:25:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:25:29.157000 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea077af-e9ca-4267-8be6-7d828615a1dc None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:25:29.157398 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea077af-e9ca-4267-8be6-7d828615a1dc None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:25:29.157547 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea077af-e9ca-4267-8be6-7d828615a1dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:25:29.225452 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-eea077af-e9ca-4267-8be6-7d828615a1dc None None] MFA Rules not processed for user `80c0c58af04e42a6b66aa1ea61117907`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:25:29.226708 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-eea077af-e9ca-4267-8be6-7d828615a1dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:25:29.281088 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:25:29.325050 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 497/993] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:25:29 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:25:29.613497 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab2ff3f-980c-4db2-9a8e-3bba4ddbbaf8 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:25:29.615035 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-1ab2ff3f-980c-4db2-9a8e-3bba4ddbbaf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:25:29.625215 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:25:29.717312 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab2ff3f-980c-4db2-9a8e-3bba4ddbbaf8 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:25:29.788996 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b749b63-879c-40a0-8ddd-592c634f485b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:25:29.789594 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b749b63-879c-40a0-8ddd-592c634f485b service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:25:29.789869 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b749b63-879c-40a0-8ddd-592c634f485b service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:25:29.790136 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b749b63-879c-40a0-8ddd-592c634f485b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:25:29.791692 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-7b749b63-879c-40a0-8ddd-592c634f485b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:25:29.851817 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:25:29.898172 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b749b63-879c-40a0-8ddd-592c634f485b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:25:29.899592 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b749b63-879c-40a0-8ddd-592c634f485b service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:25:29.917432 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 497/994] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:25:29 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:25:32.594565 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e280fb4d-3a12-4b78-b6a9-8a4a3d3027a6 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:25:32.595081 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e280fb4d-3a12-4b78-b6a9-8a4a3d3027a6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:25:32.595172 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e280fb4d-3a12-4b78-b6a9-8a4a3d3027a6 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:25:32.596486 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 498/995] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:25:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:25:32.601831 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc6427-a3f3-4e2e-9895-bdd3e312fc0f None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:25:32.602198 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc6427-a3f3-4e2e-9895-bdd3e312fc0f None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:25:32.602378 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fc6427-a3f3-4e2e-9895-bdd3e312fc0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:25:32.667611 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-e4fc6427-a3f3-4e2e-9895-bdd3e312fc0f None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:25:32.668842 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e4fc6427-a3f3-4e2e-9895-bdd3e312fc0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:25:32.763608 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 498/996] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:25:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:25:33.042527 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e280fb4d-3a12-4b78-b6a9-8a4a3d3027a6 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:25:33.044098 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e280fb4d-3a12-4b78-b6a9-8a4a3d3027a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:25:33.144330 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e280fb4d-3a12-4b78-b6a9-8a4a3d3027a6 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:25:33.215763 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c191863-6af3-4284-a749-545e205e15e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:25:33.216400 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c191863-6af3-4284-a749-545e205e15e6 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:25:33.216679 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c191863-6af3-4284-a749-545e205e15e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:25:33.216951 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c191863-6af3-4284-a749-545e205e15e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:25:33.218528 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5c191863-6af3-4284-a749-545e205e15e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:25:33.225222 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:25:33.328551 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c191863-6af3-4284-a749-545e205e15e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:25:33.329887 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c191863-6af3-4284-a749-545e205e15e6 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:25:33.347329 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 499/997] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:25:33 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:25:36.134620 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d9fec9-9001-4af3-91ac-034ff0d4f5b0 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:25:36.135144 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d9fec9-9001-4af3-91ac-034ff0d4f5b0 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:25:36.135144 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d9fec9-9001-4af3-91ac-034ff0d4f5b0 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:25:36.136703 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 499/998] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:25:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:25:36.142904 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f18486-3426-48c6-9069-e8d64108f2f1 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:25:36.143481 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f18486-3426-48c6-9069-e8d64108f2f1 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:25:36.143481 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f18486-3426-48c6-9069-e8d64108f2f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:25:36.215625 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a8f18486-3426-48c6-9069-e8d64108f2f1 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:25:36.216893 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a8f18486-3426-48c6-9069-e8d64108f2f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:25:36.278698 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:25:36.319417 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 500/999] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:25:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:25:36.602489 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d9fec9-9001-4af3-91ac-034ff0d4f5b0 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:25:36.604146 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-57d9fec9-9001-4af3-91ac-034ff0d4f5b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:25:36.624813 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:25:36.711613 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d9fec9-9001-4af3-91ac-034ff0d4f5b0 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:25:36.788701 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c79dcc6-1ce4-47c2-9d0b-be3f83f66673 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:25:36.789234 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c79dcc6-1ce4-47c2-9d0b-be3f83f66673 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:25:36.789518 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c79dcc6-1ce4-47c2-9d0b-be3f83f66673 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:25:36.789792 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c79dcc6-1ce4-47c2-9d0b-be3f83f66673 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:25:36.791382 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-3c79dcc6-1ce4-47c2-9d0b-be3f83f66673 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:25:36.867189 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:25:36.905326 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c79dcc6-1ce4-47c2-9d0b-be3f83f66673 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:25:36.906762 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c79dcc6-1ce4-47c2-9d0b-be3f83f66673 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:25:36.925144 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 500/1000] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:25:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:25:39.133255 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52be81-8813-4f04-8523-d420ffe6aae2 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:25:39.133803 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52be81-8813-4f04-8523-d420ffe6aae2 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:25:39.133803 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad52be81-8813-4f04-8523-d420ffe6aae2 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:25:39.135149 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 501/1001] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:25:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:25:39.140890 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94670a7-ac57-421c-a6ba-9893c840e462 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:25:39.141311 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94670a7-ac57-421c-a6ba-9893c840e462 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:25:39.141440 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94670a7-ac57-421c-a6ba-9893c840e462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:25:39.216113 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-f94670a7-ac57-421c-a6ba-9893c840e462 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:25:39.217940 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f94670a7-ac57-421c-a6ba-9893c840e462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:25:39.316043 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 501/1002] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:25:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:25:39.594294 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad52be81-8813-4f04-8523-d420ffe6aae2 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:25:39.595985 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ad52be81-8813-4f04-8523-d420ffe6aae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:25:39.700101 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad52be81-8813-4f04-8523-d420ffe6aae2 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:25:39.772571 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aabe57-0555-49bf-9b69-e337499ffc7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:25:39.773180 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aabe57-0555-49bf-9b69-e337499ffc7c service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:25:39.773456 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aabe57-0555-49bf-9b69-e337499ffc7c service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:25:39.773726 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aabe57-0555-49bf-9b69-e337499ffc7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:25:39.775300 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a1aabe57-0555-49bf-9b69-e337499ffc7c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:25:39.792422 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:25:39.885829 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1aabe57-0555-49bf-9b69-e337499ffc7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:25:39.887266 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1aabe57-0555-49bf-9b69-e337499ffc7c service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:25:39.906072 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 502/1003] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:25:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:25:42.094992 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8316e6-d97f-4f32-8490-5fb5430919dc None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:25:42.095520 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8316e6-d97f-4f32-8490-5fb5430919dc None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:25:42.095520 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8316e6-d97f-4f32-8490-5fb5430919dc None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:25:42.096918 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 502/1004] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:25:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:25:42.102456 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bced20-50e8-4cbb-bb3c-0d7308f3b51a None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:25:42.102896 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bced20-50e8-4cbb-bb3c-0d7308f3b51a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:25:42.102991 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bced20-50e8-4cbb-bb3c-0d7308f3b51a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:25:42.178487 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-76bced20-50e8-4cbb-bb3c-0d7308f3b51a None None] MFA Rules not processed for user `80c0c58af04e42a6b66aa1ea61117907`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:25:42.179677 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-76bced20-50e8-4cbb-bb3c-0d7308f3b51a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:25:42.244849 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:25:42.276597 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 503/1005] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:25:42 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 03:25:42.558603 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8316e6-d97f-4f32-8490-5fb5430919dc None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:25:42.560259 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ce8316e6-d97f-4f32-8490-5fb5430919dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:25:42.591837 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:25:42.666592 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8316e6-d97f-4f32-8490-5fb5430919dc None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:25:42.741711 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e513f6b-a57d-4ea7-838b-785a218f7f32 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:25:42.742311 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e513f6b-a57d-4ea7-838b-785a218f7f32 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:25:42.742587 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e513f6b-a57d-4ea7-838b-785a218f7f32 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:25:42.742856 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e513f6b-a57d-4ea7-838b-785a218f7f32 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:25:42.744460 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-8e513f6b-a57d-4ea7-838b-785a218f7f32 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:25:42.820144 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:25:42.855656 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e513f6b-a57d-4ea7-838b-785a218f7f32 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:25:42.857127 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e513f6b-a57d-4ea7-838b-785a218f7f32 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:25:42.876031 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 503/1006] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:25:42 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:25:45.227983 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97478296-deaa-462c-a404-d38fc54fce91 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:25:45.228582 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97478296-deaa-462c-a404-d38fc54fce91 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:25:45.228582 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97478296-deaa-462c-a404-d38fc54fce91 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:25:45.229875 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 504/1007] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:25:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:25:45.235115 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b896e3-eb61-474a-8dfb-f5f045ea3d49 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:25:45.235519 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b896e3-eb61-474a-8dfb-f5f045ea3d49 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:25:45.235685 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b896e3-eb61-474a-8dfb-f5f045ea3d49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:25:45.300145 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-c6b896e3-eb61-474a-8dfb-f5f045ea3d49 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:25:45.301331 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-c6b896e3-eb61-474a-8dfb-f5f045ea3d49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:25:45.395957 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 504/1008] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:25:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:25:45.678188 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97478296-deaa-462c-a404-d38fc54fce91 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:25:45.679723 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-97478296-deaa-462c-a404-d38fc54fce91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:25:45.781126 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97478296-deaa-462c-a404-d38fc54fce91 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:25:45.853048 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d748932-7282-4d22-9215-e9640229d5c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:25:45.853654 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d748932-7282-4d22-9215-e9640229d5c1 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:25:45.853931 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d748932-7282-4d22-9215-e9640229d5c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:25:45.854200 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d748932-7282-4d22-9215-e9640229d5c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:25:45.855774 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7d748932-7282-4d22-9215-e9640229d5c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:25:45.882490 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:25:45.963128 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d748932-7282-4d22-9215-e9640229d5c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:25:45.964585 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d748932-7282-4d22-9215-e9640229d5c1 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:25:45.982939 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 505/1009] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:25:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:25:48.142463 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a50b10c-8adb-4a40-8574-ab4453804a20 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:25:48.142982 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a50b10c-8adb-4a40-8574-ab4453804a20 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:25:48.142982 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a50b10c-8adb-4a40-8574-ab4453804a20 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:25:48.144330 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 505/1010] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:25:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:25:48.149521 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88122d91-bde3-473d-8730-9a183cba0801 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:25:48.149936 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88122d91-bde3-473d-8730-9a183cba0801 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:25:48.150074 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88122d91-bde3-473d-8730-9a183cba0801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:25:48.214859 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-88122d91-bde3-473d-8730-9a183cba0801 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:25:48.216087 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-88122d91-bde3-473d-8730-9a183cba0801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:25:48.289863 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:25:48.311708 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 506/1011] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:25:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 03:25:48.597208 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a50b10c-8adb-4a40-8574-ab4453804a20 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:25:48.598751 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-4a50b10c-8adb-4a40-8574-ab4453804a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:25:48.638672 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:25:48.700876 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a50b10c-8adb-4a40-8574-ab4453804a20 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:25:48.774580 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972c1b01-a533-47f1-9bcb-6a083f3ff0a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:25:48.775178 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c1b01-a533-47f1-9bcb-6a083f3ff0a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:25:48.775453 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c1b01-a533-47f1-9bcb-6a083f3ff0a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:25:48.775723 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c1b01-a533-47f1-9bcb-6a083f3ff0a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:25:48.777326 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-972c1b01-a533-47f1-9bcb-6a083f3ff0a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:25:48.859067 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:25:48.885219 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972c1b01-a533-47f1-9bcb-6a083f3ff0a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:25:48.886652 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972c1b01-a533-47f1-9bcb-6a083f3ff0a6 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:25:48.905756 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 506/1012] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:25:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:25:51.343336 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1945d7a1-9e06-48e4-b34f-e8b38c1e133a None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:25:51.343893 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1945d7a1-9e06-48e4-b34f-e8b38c1e133a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:25:51.343893 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1945d7a1-9e06-48e4-b34f-e8b38c1e133a None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:25:51.345289 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 507/1013] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:25:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:25:51.350355 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaf5762-d598-4bb8-8a60-d8929098baa5 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:25:51.350788 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaf5762-d598-4bb8-8a60-d8929098baa5 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:25:51.350893 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbaf5762-d598-4bb8-8a60-d8929098baa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:25:51.415973 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-fbaf5762-d598-4bb8-8a60-d8929098baa5 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:25:51.417172 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-fbaf5762-d598-4bb8-8a60-d8929098baa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:25:51.511106 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 507/1014] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:25:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:25:51.797882 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1945d7a1-9e06-48e4-b34f-e8b38c1e133a None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:25:51.799374 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1945d7a1-9e06-48e4-b34f-e8b38c1e133a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:25:51.900415 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1945d7a1-9e06-48e4-b34f-e8b38c1e133a None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:25:51.972220 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f05f368-6a4f-495c-b2a0-c514e2d682d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:25:51.972818 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05f368-6a4f-495c-b2a0-c514e2d682d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:25:51.973094 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05f368-6a4f-495c-b2a0-c514e2d682d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:25:51.973365 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05f368-6a4f-495c-b2a0-c514e2d682d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:25:51.974929 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9f05f368-6a4f-495c-b2a0-c514e2d682d2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:25:52.009138 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:25:52.082284 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f05f368-6a4f-495c-b2a0-c514e2d682d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:25:52.083628 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f05f368-6a4f-495c-b2a0-c514e2d682d2 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:25:52.102011 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 508/1015] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:25:51 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:25:56.176789 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a5c3cf-13a5-44da-aa2f-5ef0a22f39d6 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:25:56.177297 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a5c3cf-13a5-44da-aa2f-5ef0a22f39d6 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:25:56.177297 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a5c3cf-13a5-44da-aa2f-5ef0a22f39d6 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:25:56.178637 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 508/1016] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:25:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:25:56.184294 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c6e4e-359a-40f7-b914-e3b153a1a59e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:25:56.184690 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c6e4e-359a-40f7-b914-e3b153a1a59e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:25:56.184799 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96c6e4e-359a-40f7-b914-e3b153a1a59e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:25:56.253313 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-c96c6e4e-359a-40f7-b914-e3b153a1a59e None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:25:56.254505 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c96c6e4e-359a-40f7-b914-e3b153a1a59e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:25:56.339689 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:25:56.351657 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 509/1017] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:25:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:25:56.637436 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a5c3cf-13a5-44da-aa2f-5ef0a22f39d6 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:25:56.639016 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-09a5c3cf-13a5-44da-aa2f-5ef0a22f39d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:25:56.689190 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:25:56.745974 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a5c3cf-13a5-44da-aa2f-5ef0a22f39d6 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:25:56.821606 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16395d91-86dc-4b00-a761-fe2ddae0140b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:25:56.822135 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16395d91-86dc-4b00-a761-fe2ddae0140b service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:25:56.822414 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16395d91-86dc-4b00-a761-fe2ddae0140b service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:25:56.822687 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16395d91-86dc-4b00-a761-fe2ddae0140b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:25:56.824303 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-16395d91-86dc-4b00-a761-fe2ddae0140b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:25:56.923637 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:25:56.941232 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16395d91-86dc-4b00-a761-fe2ddae0140b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:25:56.942704 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16395d91-86dc-4b00-a761-fe2ddae0140b service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:25:56.960549 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 509/1018] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:25:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:25:59.768124 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425bd55-87ab-4d4d-98c4-01c156d2b1c7 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:25:59.768689 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425bd55-87ab-4d4d-98c4-01c156d2b1c7 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:25:59.768791 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a425bd55-87ab-4d4d-98c4-01c156d2b1c7 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:25:59.770158 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 510/1019] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:25:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:25:59.775416 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfbf7e0-a10b-4be1-b87a-b21223d820e9 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:25:59.775849 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfbf7e0-a10b-4be1-b87a-b21223d820e9 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:25:59.775992 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfbf7e0-a10b-4be1-b87a-b21223d820e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:25:59.844074 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-8cfbf7e0-a10b-4be1-b87a-b21223d820e9 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:25:59.845285 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-8cfbf7e0-a10b-4be1-b87a-b21223d820e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:25:59.946039 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 510/1020] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:25:59 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:26:00.244328 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06990dd2-d392-4dc8-a7e1-9dc19d3f3079 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:00.244614 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06990dd2-d392-4dc8-a7e1-9dc19d3f3079 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:00.244907 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06990dd2-d392-4dc8-a7e1-9dc19d3f3079 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:00.246329 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 511/1021] 10.222.0.26 () {54 vars in 965 bytes} [Wed Nov 2 03:26:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:26:00.253168 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31999358-4a67-4d68-8a07-a1e746728788 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:00.253538 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31999358-4a67-4d68-8a07-a1e746728788 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:00.253883 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31999358-4a67-4d68-8a07-a1e746728788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:00.325166 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:26:00.334950 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-31999358-4a67-4d68-8a07-a1e746728788 None None] MFA Rules not processed for user `f07e1b51b00f42b7b74c3689268635cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:26:00.336233 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-31999358-4a67-4d68-8a07-a1e746728788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:26:00.413109 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 511/1022] 10.222.0.26 () {58 vars in 1062 bytes} [Wed Nov 2 03:26:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:26:00.422235 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06990dd2-d392-4dc8-a7e1-9dc19d3f3079 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:26:00.423762 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-06990dd2-d392-4dc8-a7e1-9dc19d3f3079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:26:00.530907 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06990dd2-d392-4dc8-a7e1-9dc19d3f3079 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:26:00.617905 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e023d3-d6d5-48cd-bc5f-5323f45db3ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:26:00.618517 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e023d3-d6d5-48cd-bc5f-5323f45db3ab service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:00.618794 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e023d3-d6d5-48cd-bc5f-5323f45db3ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:00.619065 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e023d3-d6d5-48cd-bc5f-5323f45db3ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:00.620688 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-09e023d3-d6d5-48cd-bc5f-5323f45db3ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:26:00.668327 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:26:00.745591 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e023d3-d6d5-48cd-bc5f-5323f45db3ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:26:00.747035 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e023d3-d6d5-48cd-bc5f-5323f45db3ab service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:26:00.767273 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 512/1023] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:26:00 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:26:05.569848 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff798c8-4b03-4640-862c-9b5fbf2622e6 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:05.570358 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff798c8-4b03-4640-862c-9b5fbf2622e6 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:05.570543 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff798c8-4b03-4640-862c-9b5fbf2622e6 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:05.572104 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 512/1024] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:26:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:26:05.577993 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c8730b-7659-4a7b-bf60-9393231215ad None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:05.578724 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c8730b-7659-4a7b-bf60-9393231215ad None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:05.578724 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c8730b-7659-4a7b-bf60-9393231215ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:05.647801 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a2c8730b-7659-4a7b-bf60-9393231215ad None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:26:05.649109 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a2c8730b-7659-4a7b-bf60-9393231215ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:26:05.744528 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:26:05.746863 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 513/1025] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:26:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:26:06.032148 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff798c8-4b03-4640-862c-9b5fbf2622e6 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:26:06.033749 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-2ff798c8-4b03-4640-862c-9b5fbf2622e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:26:06.140835 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff798c8-4b03-4640-862c-9b5fbf2622e6 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:26:06.187619 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:26:06.213161 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604947d5-9277-4667-9330-793c9df9fec9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:26:06.214085 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604947d5-9277-4667-9330-793c9df9fec9 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:06.214362 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604947d5-9277-4667-9330-793c9df9fec9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:06.214631 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604947d5-9277-4667-9330-793c9df9fec9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:06.216249 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-604947d5-9277-4667-9330-793c9df9fec9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:26:06.323621 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604947d5-9277-4667-9330-793c9df9fec9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:26:06.324996 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604947d5-9277-4667-9330-793c9df9fec9 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:26:06.342383 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 513/1026] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:26:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:26:11.199604 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1204c29e-312d-4931-99b9-39c4de1f9c5c None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:11.201137 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1204c29e-312d-4931-99b9-39c4de1f9c5c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:11.201621 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1204c29e-312d-4931-99b9-39c4de1f9c5c None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:11.203564 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 514/1027] 10.222.0.26 () {54 vars in 921 bytes} [Wed Nov 2 03:26:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:26:11.208348 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c1c724-47e5-4640-8ebf-4a8b91d62cc5 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:11.208739 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c1c724-47e5-4640-8ebf-4a8b91d62cc5 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:11.208899 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c1c724-47e5-4640-8ebf-4a8b91d62cc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:11.341405 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-38c1c724-47e5-4640-8ebf-4a8b91d62cc5 None None] MFA Rules not processed for user `9b4b9bd741574ddabadd4b7651ebc885`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:26:11.342687 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-38c1c724-47e5-4640-8ebf-4a8b91d62cc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:26:11.369401 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:26:11.466551 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 514/1028] 10.222.0.26 () {58 vars in 1018 bytes} [Wed Nov 2 03:26:11 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:26:15.527861 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4c7334-cec1-4a98-ad90-a005e4dd1af7 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:15.529491 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4c7334-cec1-4a98-ad90-a005e4dd1af7 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:15.529994 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4c7334-cec1-4a98-ad90-a005e4dd1af7 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:15.531816 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 515/1029] 10.222.0.26 () {54 vars in 921 bytes} [Wed Nov 2 03:26:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:26:15.537520 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71334be5-75e2-4164-997a-f5c23b3a3883 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:15.538002 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71334be5-75e2-4164-997a-f5c23b3a3883 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:15.538084 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71334be5-75e2-4164-997a-f5c23b3a3883 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:15.618373 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-71334be5-75e2-4164-997a-f5c23b3a3883 None None] MFA Rules not processed for user `9b4b9bd741574ddabadd4b7651ebc885`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:26:15.619579 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-71334be5-75e2-4164-997a-f5c23b3a3883 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:26:15.694782 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:26:15.733378 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 515/1030] 10.222.0.26 () {58 vars in 1018 bytes} [Wed Nov 2 03:26:15 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 03:26:17.189819 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f11fed3-6f40-434b-b4ce-5383d066f6de None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:17.190395 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f11fed3-6f40-434b-b4ce-5383d066f6de None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:17.190395 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f11fed3-6f40-434b-b4ce-5383d066f6de None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:17.191857 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 516/1031] 10.222.0.26 () {54 vars in 921 bytes} [Wed Nov 2 03:26:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:26:17.197076 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff147e4-4e52-4bac-aa2f-610b10614b69 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:17.197515 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff147e4-4e52-4bac-aa2f-610b10614b69 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:17.197610 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff147e4-4e52-4bac-aa2f-610b10614b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:17.276543 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-fff147e4-4e52-4bac-aa2f-610b10614b69 None None] MFA Rules not processed for user `9b4b9bd741574ddabadd4b7651ebc885`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:26:17.277761 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-fff147e4-4e52-4bac-aa2f-610b10614b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:26:17.384314 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 516/1032] 10.222.0.26 () {58 vars in 1018 bytes} [Wed Nov 2 03:26:17 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:26:20.925011 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d6153-d2c0-4d61-93f0-576e695a094b None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:20.925500 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d6153-d2c0-4d61-93f0-576e695a094b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:20.925500 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8d6153-d2c0-4d61-93f0-576e695a094b None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:20.926801 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 517/1033] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:26:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:26:20.932137 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866b2240-0fd5-40d2-9a45-915193ae8a99 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:20.932549 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866b2240-0fd5-40d2-9a45-915193ae8a99 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:20.932728 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866b2240-0fd5-40d2-9a45-915193ae8a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:20.972810 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:26:21.006663 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-866b2240-0fd5-40d2-9a45-915193ae8a99 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:26:21.007853 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-866b2240-0fd5-40d2-9a45-915193ae8a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:26:21.111528 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 517/1034] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:26:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:26:21.431807 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7de9d0-bd43-4f69-9626-acebaada7640 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:21.432402 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7de9d0-bd43-4f69-9626-acebaada7640 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:21.433049 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7de9d0-bd43-4f69-9626-acebaada7640 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:21.434993 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 518/1035] 10.222.0.26 () {54 vars in 961 bytes} [Wed Nov 2 03:26:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:26:21.442736 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43523353-4256-4a2a-8b0e-c57daffb8843 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:21.443446 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43523353-4256-4a2a-8b0e-c57daffb8843 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:21.443878 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43523353-4256-4a2a-8b0e-c57daffb8843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:21.531705 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-43523353-4256-4a2a-8b0e-c57daffb8843 None None] MFA Rules not processed for user `44961b06151c4da382fcd54dd1f1e146`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:26:21.533083 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-43523353-4256-4a2a-8b0e-c57daffb8843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:26:21.586753 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:26:21.664177 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 518/1036] 10.222.0.26 () {58 vars in 1058 bytes} [Wed Nov 2 03:26:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 03:26:21.673094 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7de9d0-bd43-4f69-9626-acebaada7640 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:26:21.675568 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9f7de9d0-bd43-4f69-9626-acebaada7640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:26:21.846683 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7de9d0-bd43-4f69-9626-acebaada7640 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:26:21.925779 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:26:21.969351 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eb0903-0eec-44c8-ba0a-da78e35f9735 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:26:21.970084 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb0903-0eec-44c8-ba0a-da78e35f9735 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:21.970432 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb0903-0eec-44c8-ba0a-da78e35f9735 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:21.970776 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eb0903-0eec-44c8-ba0a-da78e35f9735 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:21.972466 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f2eb0903-0eec-44c8-ba0a-da78e35f9735 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:26:22.078943 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eb0903-0eec-44c8-ba0a-da78e35f9735 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:26:22.081354 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eb0903-0eec-44c8-ba0a-da78e35f9735 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:26:22.102788 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 519/1037] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:26:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:26:24.484954 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16818725-e5fd-4fc2-8490-410ec50ebf73 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:24.485477 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16818725-e5fd-4fc2-8490-410ec50ebf73 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:24.485477 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16818725-e5fd-4fc2-8490-410ec50ebf73 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:24.487124 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 519/1038] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:26:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:26:24.492565 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbfb43f-502a-4f2e-af82-e235c0d746e4 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:24.493015 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbfb43f-502a-4f2e-af82-e235c0d746e4 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:24.493139 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbfb43f-502a-4f2e-af82-e235c0d746e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:24.598281 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-7fbfb43f-502a-4f2e-af82-e235c0d746e4 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:26:24.599559 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7fbfb43f-502a-4f2e-af82-e235c0d746e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:26:24.639388 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:26:24.707612 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 520/1039] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:26:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:26:24.994235 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16818725-e5fd-4fc2-8490-410ec50ebf73 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:26:24.995808 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-16818725-e5fd-4fc2-8490-410ec50ebf73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:26:25.168649 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16818725-e5fd-4fc2-8490-410ec50ebf73 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:26:25.180820 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:26:25.305280 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061fdd93-cfbe-4bb9-bd66-981147481ef1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:26:25.305890 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061fdd93-cfbe-4bb9-bd66-981147481ef1 service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:25.306169 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061fdd93-cfbe-4bb9-bd66-981147481ef1 service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:25.306444 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061fdd93-cfbe-4bb9-bd66-981147481ef1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:25.308050 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-061fdd93-cfbe-4bb9-bd66-981147481ef1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:26:25.403871 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061fdd93-cfbe-4bb9-bd66-981147481ef1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:26:25.405490 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061fdd93-cfbe-4bb9-bd66-981147481ef1 service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:26:25.424107 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 520/1040] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:26:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:26:27.897099 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afebd951-40ef-44aa-a468-803cc46b0591 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:27.897643 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afebd951-40ef-44aa-a468-803cc46b0591 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:27.897643 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afebd951-40ef-44aa-a468-803cc46b0591 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:27.898980 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 521/1041] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:26:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:26:27.905321 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39842392-07ea-4b07-8ac1-2d3eec847c91 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:27.905745 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39842392-07ea-4b07-8ac1-2d3eec847c91 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:27.906001 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39842392-07ea-4b07-8ac1-2d3eec847c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:27.921787 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:26:27.975758 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-39842392-07ea-4b07-8ac1-2d3eec847c91 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:26:27.976998 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-39842392-07ea-4b07-8ac1-2d3eec847c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:26:28.079741 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 521/1042] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:26:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:26:28.370279 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8da463-8216-4322-80b1-1bdf05e781a4 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:28.370562 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8da463-8216-4322-80b1-1bdf05e781a4 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:28.370834 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8da463-8216-4322-80b1-1bdf05e781a4 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:28.372242 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 522/1043] 10.222.0.26 () {54 vars in 961 bytes} [Wed Nov 2 03:26:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:26:28.377622 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91df37a-57df-4335-8c8c-7dd2d8e36974 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:28.377912 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91df37a-57df-4335-8c8c-7dd2d8e36974 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:28.378193 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91df37a-57df-4335-8c8c-7dd2d8e36974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:28.455231 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-a91df37a-57df-4335-8c8c-7dd2d8e36974 None None] MFA Rules not processed for user `44961b06151c4da382fcd54dd1f1e146`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:26:28.456463 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a91df37a-57df-4335-8c8c-7dd2d8e36974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:26:28.492360 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:26:28.587821 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 522/1044] 10.222.0.26 () {58 vars in 1058 bytes} [Wed Nov 2 03:26:28 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:26:28.596295 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8da463-8216-4322-80b1-1bdf05e781a4 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:26:28.597939 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3f8da463-8216-4322-80b1-1bdf05e781a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:26:28.760202 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8da463-8216-4322-80b1-1bdf05e781a4 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:26:28.783908 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:26:28.885842 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b5e394-45ec-4051-bc4c-d12d3a63ad07 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:26:28.886454 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b5e394-45ec-4051-bc4c-d12d3a63ad07 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:28.886847 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b5e394-45ec-4051-bc4c-d12d3a63ad07 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:28.887648 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b5e394-45ec-4051-bc4c-d12d3a63ad07 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:28.889275 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e7b5e394-45ec-4051-bc4c-d12d3a63ad07 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:26:28.986323 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b5e394-45ec-4051-bc4c-d12d3a63ad07 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:26:28.988310 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b5e394-45ec-4051-bc4c-d12d3a63ad07 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:26:29.008669 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 523/1045] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:26:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:26:29.188970 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d645e1-0d70-47d2-9bff-e20b1a7bd52f None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:29.189514 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d645e1-0d70-47d2-9bff-e20b1a7bd52f None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:29.189597 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d645e1-0d70-47d2-9bff-e20b1a7bd52f None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:29.191084 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 523/1046] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:26:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:26:29.197053 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20daaf8-8f11-450d-a93d-c222d4b7341e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:29.197371 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20daaf8-8f11-450d-a93d-c222d4b7341e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:29.197649 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20daaf8-8f11-450d-a93d-c222d4b7341e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:29.237678 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:26:29.286501 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d20daaf8-8f11-450d-a93d-c222d4b7341e None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:26:29.287682 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d20daaf8-8f11-450d-a93d-c222d4b7341e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:26:29.393233 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 524/1047] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:26:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:26:29.726682 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985420d-bf34-43b4-92fa-d002236fab99 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:29.726682 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985420d-bf34-43b4-92fa-d002236fab99 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:29.727635 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2985420d-bf34-43b4-92fa-d002236fab99 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:29.731998 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 524/1048] 10.222.0.26 () {54 vars in 963 bytes} [Wed Nov 2 03:26:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:26:29.734231 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f3b29f-bc33-40c8-8a12-93846ec0515d None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:29.734593 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f3b29f-bc33-40c8-8a12-93846ec0515d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:29.734940 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f3b29f-bc33-40c8-8a12-93846ec0515d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:29.818763 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-15f3b29f-bc33-40c8-8a12-93846ec0515d None None] MFA Rules not processed for user `df541db335eb42dc8b36cc858fa7c22f`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:26:29.820036 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-15f3b29f-bc33-40c8-8a12-93846ec0515d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:26:29.869190 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:26:29.899883 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 525/1049] 10.222.0.26 () {58 vars in 1060 bytes} [Wed Nov 2 03:26:29 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:26:29.907879 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2985420d-bf34-43b4-92fa-d002236fab99 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:26:29.909409 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-2985420d-bf34-43b4-92fa-d002236fab99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:26:30.030871 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2985420d-bf34-43b4-92fa-d002236fab99 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:26:30.079758 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:26:30.117437 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58da7d5-6187-489e-8cf4-e4640a4163d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:26:30.118046 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58da7d5-6187-489e-8cf4-e4640a4163d2 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:30.118305 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58da7d5-6187-489e-8cf4-e4640a4163d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:30.118575 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58da7d5-6187-489e-8cf4-e4640a4163d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:30.120173 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a58da7d5-6187-489e-8cf4-e4640a4163d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:26:30.245052 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58da7d5-6187-489e-8cf4-e4640a4163d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:26:30.246934 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58da7d5-6187-489e-8cf4-e4640a4163d2 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:26:30.272002 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 525/1050] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:26:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:26:31.256579 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2bdf8-51e5-4e80-8f59-e6d0cc74dada None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:31.257135 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2bdf8-51e5-4e80-8f59-e6d0cc74dada None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:31.257135 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb2bdf8-51e5-4e80-8f59-e6d0cc74dada None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:31.258462 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 526/1051] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:26:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:26:31.263899 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93933be5-857c-49d3-8e7b-501d803d0f41 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:31.264388 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93933be5-857c-49d3-8e7b-501d803d0f41 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:31.264495 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93933be5-857c-49d3-8e7b-501d803d0f41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:31.330360 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-93933be5-857c-49d3-8e7b-501d803d0f41 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:26:31.331559 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-93933be5-857c-49d3-8e7b-501d803d0f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:26:31.353843 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:26:31.430997 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 526/1052] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:26:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:26:31.722990 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb2bdf8-51e5-4e80-8f59-e6d0cc74dada None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:26:31.724687 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7eb2bdf8-51e5-4e80-8f59-e6d0cc74dada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:26:31.893088 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb2bdf8-51e5-4e80-8f59-e6d0cc74dada None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:26:31.948828 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:26:32.012398 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee38196-bc5c-4477-88bd-881f125045a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:26:32.013017 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee38196-bc5c-4477-88bd-881f125045a9 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:32.013295 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee38196-bc5c-4477-88bd-881f125045a9 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:32.013564 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee38196-bc5c-4477-88bd-881f125045a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:32.015172 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-fee38196-bc5c-4477-88bd-881f125045a9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:26:32.108586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee38196-bc5c-4477-88bd-881f125045a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:26:32.110133 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee38196-bc5c-4477-88bd-881f125045a9 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:26:32.129965 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 527/1053] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:26:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:26:34.345734 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae5c6e-ef67-4e29-b3e8-8748d8d46718 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:34.346254 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae5c6e-ef67-4e29-b3e8-8748d8d46718 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:34.346254 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ae5c6e-ef67-4e29-b3e8-8748d8d46718 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:34.347690 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 527/1054] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:26:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:26:34.352905 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d2234b-7a5d-45fe-8bc3-d915d344451a None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:34.353309 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d2234b-7a5d-45fe-8bc3-d915d344451a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:34.353500 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d2234b-7a5d-45fe-8bc3-d915d344451a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:34.422043 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-18d2234b-7a5d-45fe-8bc3-d915d344451a None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:26:34.423317 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-18d2234b-7a5d-45fe-8bc3-d915d344451a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:26:34.430981 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:26:34.541528 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 528/1055] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:26:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:26:34.831741 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ae5c6e-ef67-4e29-b3e8-8748d8d46718 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:26:34.833542 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-50ae5c6e-ef67-4e29-b3e8-8748d8d46718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:26:34.989897 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ae5c6e-ef67-4e29-b3e8-8748d8d46718 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:26:34.994120 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:26:35.110231 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bdbca0-4792-46dc-83b7-b2ff7c6fc8d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:26:35.111186 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bdbca0-4792-46dc-83b7-b2ff7c6fc8d0 service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:35.111561 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bdbca0-4792-46dc-83b7-b2ff7c6fc8d0 service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:35.111939 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bdbca0-4792-46dc-83b7-b2ff7c6fc8d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:35.113671 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-58bdbca0-4792-46dc-83b7-b2ff7c6fc8d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:26:35.239385 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bdbca0-4792-46dc-83b7-b2ff7c6fc8d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:26:35.241639 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bdbca0-4792-46dc-83b7-b2ff7c6fc8d0 service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:26:35.268132 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 528/1056] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:26:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:26:37.476343 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72acc92e-d1ff-4d72-a693-d10df9e68255 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:37.476800 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72acc92e-d1ff-4d72-a693-d10df9e68255 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:37.476987 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72acc92e-d1ff-4d72-a693-d10df9e68255 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:37.478341 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 529/1057] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:26:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:26:37.483426 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f375cde9-6241-484e-b4df-0edfa6a8644a None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:37.483820 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f375cde9-6241-484e-b4df-0edfa6a8644a None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:37.484021 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f375cde9-6241-484e-b4df-0edfa6a8644a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:37.496739 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:26:37.555228 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-f375cde9-6241-484e-b4df-0edfa6a8644a None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:26:37.556423 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f375cde9-6241-484e-b4df-0edfa6a8644a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:26:37.697963 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 529/1058] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:26:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:26:38.018267 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72acc92e-d1ff-4d72-a693-d10df9e68255 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:26:38.020391 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-72acc92e-d1ff-4d72-a693-d10df9e68255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:26:38.180551 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:26:38.192306 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72acc92e-d1ff-4d72-a693-d10df9e68255 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:26:38.314937 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf67de6a-8032-445c-b42c-a8a9fc219e10 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:26:38.315562 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67de6a-8032-445c-b42c-a8a9fc219e10 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:38.315829 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67de6a-8032-445c-b42c-a8a9fc219e10 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:38.316135 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf67de6a-8032-445c-b42c-a8a9fc219e10 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:38.317739 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-cf67de6a-8032-445c-b42c-a8a9fc219e10 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:26:38.408949 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf67de6a-8032-445c-b42c-a8a9fc219e10 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:26:38.410540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf67de6a-8032-445c-b42c-a8a9fc219e10 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:26:38.420630 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:26:38.431442 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 530/1059] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:26:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:26:40.637223 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f980e93f-b9e2-4838-9458-a0e42c03291f None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:40.637818 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f980e93f-b9e2-4838-9458-a0e42c03291f None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:40.637818 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f980e93f-b9e2-4838-9458-a0e42c03291f None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:40.639031 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 530/1060] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:26:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 03:26:40.645863 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99edd694-bb4f-48a7-84e3-d3efa372ff7b None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:40.646276 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99edd694-bb4f-48a7-84e3-d3efa372ff7b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:40.646484 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99edd694-bb4f-48a7-84e3-d3efa372ff7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:40.718328 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-99edd694-bb4f-48a7-84e3-d3efa372ff7b None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:26:40.719536 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-99edd694-bb4f-48a7-84e3-d3efa372ff7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:26:40.817043 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 531/1061] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:26:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:26:41.173760 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f980e93f-b9e2-4838-9458-a0e42c03291f None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:26:41.175896 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f980e93f-b9e2-4838-9458-a0e42c03291f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:26:41.286987 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:26:41.362760 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f980e93f-b9e2-4838-9458-a0e42c03291f None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:26:41.487821 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8132a65a-6abc-47e4-bea8-b54a5eaf37ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:26:41.488893 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8132a65a-6abc-47e4-bea8-b54a5eaf37ea service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:41.489238 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8132a65a-6abc-47e4-bea8-b54a5eaf37ea service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:41.489581 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8132a65a-6abc-47e4-bea8-b54a5eaf37ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:41.491750 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-8132a65a-6abc-47e4-bea8-b54a5eaf37ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:26:41.551251 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:26:41.593160 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8132a65a-6abc-47e4-bea8-b54a5eaf37ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:26:41.594772 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8132a65a-6abc-47e4-bea8-b54a5eaf37ea service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:26:41.613323 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 531/1062] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:26:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:26:43.756338 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2566cd-3fcd-4cbb-9ba0-77ac8b5c7d17 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:43.756886 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2566cd-3fcd-4cbb-9ba0-77ac8b5c7d17 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:43.756886 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2566cd-3fcd-4cbb-9ba0-77ac8b5c7d17 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:43.758473 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 532/1063] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:26:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:26:43.764260 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcec6de-2e80-4631-8755-49220018e054 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:43.764770 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcec6de-2e80-4631-8755-49220018e054 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:43.764879 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcec6de-2e80-4631-8755-49220018e054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:43.831032 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-4dcec6de-2e80-4631-8755-49220018e054 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:26:43.832182 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-4dcec6de-2e80-4631-8755-49220018e054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:26:43.931514 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 532/1064] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:26:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:26:44.220680 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2566cd-3fcd-4cbb-9ba0-77ac8b5c7d17 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:26:44.222208 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-fa2566cd-3fcd-4cbb-9ba0-77ac8b5c7d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:26:44.299880 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:26:44.369313 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2566cd-3fcd-4cbb-9ba0-77ac8b5c7d17 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:26:44.495618 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec79deab-b8bf-479a-868f-0545e534050f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:26:44.496289 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec79deab-b8bf-479a-868f-0545e534050f service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:44.496567 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec79deab-b8bf-479a-868f-0545e534050f service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:44.496834 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec79deab-b8bf-479a-868f-0545e534050f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:44.498416 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ec79deab-b8bf-479a-868f-0545e534050f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:26:44.539546 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:26:44.593093 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec79deab-b8bf-479a-868f-0545e534050f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:26:44.594678 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec79deab-b8bf-479a-868f-0545e534050f service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:26:44.613354 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 533/1065] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:26:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:26:45.450394 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414f1bf7-d016-45a2-8536-14694d9ceed8 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:45.450952 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414f1bf7-d016-45a2-8536-14694d9ceed8 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:45.450952 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414f1bf7-d016-45a2-8536-14694d9ceed8 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:45.452347 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 533/1066] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:26:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:26:45.457515 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0145cd72-351b-47c9-8116-9034a24605d4 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:45.457947 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0145cd72-351b-47c9-8116-9034a24605d4 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:45.458046 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0145cd72-351b-47c9-8116-9034a24605d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:45.524959 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-0145cd72-351b-47c9-8116-9034a24605d4 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:26:45.526228 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0145cd72-351b-47c9-8116-9034a24605d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:26:45.627147 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 534/1067] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:26:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:26:45.929356 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326d5e4-faf8-47a7-9e1b-b217e78d6482 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:45.929637 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326d5e4-faf8-47a7-9e1b-b217e78d6482 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:45.929907 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326d5e4-faf8-47a7-9e1b-b217e78d6482 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:45.931372 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 534/1068] 10.222.0.26 () {54 vars in 962 bytes} [Wed Nov 2 03:26:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:26:45.938112 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e071584-6458-4323-8c5f-1b1ab5632aeb None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:45.938681 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e071584-6458-4323-8c5f-1b1ab5632aeb None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:45.939058 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e071584-6458-4323-8c5f-1b1ab5632aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:45.957144 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:26:46.022835 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-0e071584-6458-4323-8c5f-1b1ab5632aeb None None] MFA Rules not processed for user `50aab624ee9c4651a7cd42124bcc465f`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:26:46.024061 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0e071584-6458-4323-8c5f-1b1ab5632aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:26:46.098574 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 535/1069] 10.222.0.26 () {58 vars in 1059 bytes} [Wed Nov 2 03:26:45 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:26:46.106937 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e326d5e4-faf8-47a7-9e1b-b217e78d6482 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:26:46.108628 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e326d5e4-faf8-47a7-9e1b-b217e78d6482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:26:46.140891 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:26:46.214388 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e326d5e4-faf8-47a7-9e1b-b217e78d6482 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:26:46.289967 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2176607-6f94-4cf8-b6e2-a965e67e49d4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50aab624ee9c4651a7cd42124bcc465f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:26:46.290568 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2176607-6f94-4cf8-b6e2-a965e67e49d4 service glance] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:46.290842 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2176607-6f94-4cf8-b6e2-a965e67e49d4 service glance] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:46.291116 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2176607-6f94-4cf8-b6e2-a965e67e49d4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:46.292776 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d2176607-6f94-4cf8-b6e2-a965e67e49d4 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:26:46.369693 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:26:46.402643 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2176607-6f94-4cf8-b6e2-a965e67e49d4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:26:46.404135 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2176607-6f94-4cf8-b6e2-a965e67e49d4 service glance] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:26:46.422352 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 535/1070] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:26:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:26:46.823213 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe569d24-a4e7-49f4-b4bc-b3679157a65a None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:46.823807 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe569d24-a4e7-49f4-b4bc-b3679157a65a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:46.823807 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe569d24-a4e7-49f4-b4bc-b3679157a65a None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:46.825182 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 536/1071] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:26:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:26:46.830525 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb810c-5fe1-4571-8ecb-34c414e560f4 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:46.830912 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb810c-5fe1-4571-8ecb-34c414e560f4 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:46.831072 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acb810c-5fe1-4571-8ecb-34c414e560f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:46.901328 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-9acb810c-5fe1-4571-8ecb-34c414e560f4 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:26:46.902609 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-9acb810c-5fe1-4571-8ecb-34c414e560f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:26:47.012908 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 536/1072] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:26:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:26:47.184880 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cbb72b-971e-4877-97e5-9b3ae3d762e4 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:47.185235 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cbb72b-971e-4877-97e5-9b3ae3d762e4 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:47.185624 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cbb72b-971e-4877-97e5-9b3ae3d762e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:47.253071 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a4cbb72b-971e-4877-97e5-9b3ae3d762e4 None None] MFA Rules not processed for user `da7ee607b66e43f3823d9d2faf373974`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:26:47.254306 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a4cbb72b-971e-4877-97e5-9b3ae3d762e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:26:47.304323 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acb810c-5fe1-4571-8ecb-34c414e560f4 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:26:47.305525 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-9acb810c-5fe1-4571-8ecb-34c414e560f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:26:47.313222 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:26:47.346073 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 537/1073] 10.222.0.26 () {58 vars in 1009 bytes} [Wed Nov 2 03:26:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 03:26:47.346349 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:26:47.389713 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25611237-df17-4f92-ab20-5274de15aaba None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:47.390004 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25611237-df17-4f92-ab20-5274de15aaba None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:47.390284 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25611237-df17-4f92-ab20-5274de15aaba None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:47.391643 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 538/1074] 10.222.0.26 () {54 vars in 943 bytes} [Wed Nov 2 03:26:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 03:26:47.397792 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e5d3a2-9f5d-42da-97d6-25e2313b971e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:47.398159 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e5d3a2-9f5d-42da-97d6-25e2313b971e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:47.398497 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e5d3a2-9f5d-42da-97d6-25e2313b971e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:47.425064 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acb810c-5fe1-4571-8ecb-34c414e560f4 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:26:47.474453 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-82e5d3a2-9f5d-42da-97d6-25e2313b971e None None] MFA Rules not processed for user `5193c40a917b4105ab8b27aead1685f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:26:47.475657 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-82e5d3a2-9f5d-42da-97d6-25e2313b971e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:26:47.547115 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfb7469-7872-4a6f-ae80-5a25fdb23902 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:26:47.547735 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb7469-7872-4a6f-ae80-5a25fdb23902 service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:47.548047 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb7469-7872-4a6f-ae80-5a25fdb23902 service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:47.548323 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb7469-7872-4a6f-ae80-5a25fdb23902 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:47.550006 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ddfb7469-7872-4a6f-ae80-5a25fdb23902 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:26:47.585713 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:26:47.587182 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 539/1075] 10.222.0.26 () {58 vars in 1040 bytes} [Wed Nov 2 03:26:47 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 03:26:47.596794 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e5d3a2-9f5d-42da-97d6-25e2313b971e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:26:47.606762 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e5d3a2-9f5d-42da-97d6-25e2313b971e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:26:47.632543 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:26:47.645186 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfb7469-7872-4a6f-ae80-5a25fdb23902 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:26:47.646829 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfb7469-7872-4a6f-ae80-5a25fdb23902 service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:26:47.664749 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 537/1076] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:26:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 03:26:47.716783 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe59784-d0b7-44cc-837b-30c7fb5b90a9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5193c40a917b4105ab8b27aead1685f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:26:47.717419 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe59784-d0b7-44cc-837b-30c7fb5b90a9 service swift] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:47.717734 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe59784-d0b7-44cc-837b-30c7fb5b90a9 service swift] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:47.718010 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe59784-d0b7-44cc-837b-30c7fb5b90a9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:47.719690 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9fe59784-d0b7-44cc-837b-30c7fb5b90a9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:26:47.812651 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe59784-d0b7-44cc-837b-30c7fb5b90a9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:26:47.814274 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe59784-d0b7-44cc-837b-30c7fb5b90a9 service swift] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:26:47.823357 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 540/1077] 10.222.0.26 () {60 vars in 1384 bytes} [Wed Nov 2 03:26:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Nov 02 03:26:49.907900 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4576282-8afd-476a-a599-3711bc48a1c3 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:49.908521 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4576282-8afd-476a-a599-3711bc48a1c3 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:49.908521 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4576282-8afd-476a-a599-3711bc48a1c3 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:49.910094 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 538/1078] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:26:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:26:49.915714 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f229a08c-7f30-49b4-a97f-06febba9105c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:49.916184 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f229a08c-7f30-49b4-a97f-06febba9105c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:49.916363 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f229a08c-7f30-49b4-a97f-06febba9105c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:49.988382 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-f229a08c-7f30-49b4-a97f-06febba9105c None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:26:49.989610 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f229a08c-7f30-49b4-a97f-06febba9105c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:26:49.997222 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:26:50.115080 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 541/1079] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:26:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:26:50.399797 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4576282-8afd-476a-a599-3711bc48a1c3 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:26:50.401364 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-b4576282-8afd-476a-a599-3711bc48a1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:26:50.569310 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4576282-8afd-476a-a599-3711bc48a1c3 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:26:50.582398 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:26:50.694445 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc6b6ad-0e91-42d4-8dfe-d4f9cc79c049 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:26:50.694965 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc6b6ad-0e91-42d4-8dfe-d4f9cc79c049 service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:50.695243 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc6b6ad-0e91-42d4-8dfe-d4f9cc79c049 service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:50.697363 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc6b6ad-0e91-42d4-8dfe-d4f9cc79c049 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:50.699588 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ffc6b6ad-0e91-42d4-8dfe-d4f9cc79c049 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:26:50.798000 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc6b6ad-0e91-42d4-8dfe-d4f9cc79c049 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:26:50.799584 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc6b6ad-0e91-42d4-8dfe-d4f9cc79c049 service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:26:50.820930 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 539/1080] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:26:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:26:53.041726 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60893ca-d80d-41ae-a7c1-02bf358e0c58 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:53.042369 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60893ca-d80d-41ae-a7c1-02bf358e0c58 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:53.042369 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60893ca-d80d-41ae-a7c1-02bf358e0c58 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:53.043840 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 542/1081] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:26:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:26:53.049086 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc7258-b84f-4034-abae-540b4422cd14 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:53.049501 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc7258-b84f-4034-abae-540b4422cd14 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:53.049662 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc7258-b84f-4034-abae-540b4422cd14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:53.071245 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:26:53.135142 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-bbcc7258-b84f-4034-abae-540b4422cd14 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:26:53.137171 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-bbcc7258-b84f-4034-abae-540b4422cd14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:26:53.280206 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 540/1082] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:26:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:26:53.580526 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60893ca-d80d-41ae-a7c1-02bf358e0c58 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:26:53.582517 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d60893ca-d80d-41ae-a7c1-02bf358e0c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:26:53.735130 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:26:53.747081 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60893ca-d80d-41ae-a7c1-02bf358e0c58 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:26:53.879775 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157eff3c-e313-4744-85e3-1ccc9952694e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:26:53.880416 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157eff3c-e313-4744-85e3-1ccc9952694e service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:53.880687 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157eff3c-e313-4744-85e3-1ccc9952694e service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:53.880953 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157eff3c-e313-4744-85e3-1ccc9952694e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:53.882533 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-157eff3c-e313-4744-85e3-1ccc9952694e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:26:53.983281 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157eff3c-e313-4744-85e3-1ccc9952694e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:26:53.984916 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157eff3c-e313-4744-85e3-1ccc9952694e service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:26:53.996830 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:26:54.007087 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 543/1083] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:26:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:26:56.220559 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021f129-8c9e-4e30-bf15-cd0f251789e9 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:56.221116 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021f129-8c9e-4e30-bf15-cd0f251789e9 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:56.221116 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021f129-8c9e-4e30-bf15-cd0f251789e9 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:56.222435 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 541/1084] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:26:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:26:56.227766 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c30614-e0e3-4b4c-ad4a-e60c0be00015 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:56.228248 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c30614-e0e3-4b4c-ad4a-e60c0be00015 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:56.228358 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c30614-e0e3-4b4c-ad4a-e60c0be00015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:56.305900 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-49c30614-e0e3-4b4c-ad4a-e60c0be00015 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:26:56.307108 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-49c30614-e0e3-4b4c-ad4a-e60c0be00015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:26:56.409041 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 544/1085] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:26:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:26:56.706660 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b021f129-8c9e-4e30-bf15-cd0f251789e9 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:26:56.708546 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-b021f129-8c9e-4e30-bf15-cd0f251789e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:26:56.831828 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:26:56.909993 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b021f129-8c9e-4e30-bf15-cd0f251789e9 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:26:57.065847 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f9f638-1f0b-4e4f-b546-c636d746963a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:26:57.066498 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9f638-1f0b-4e4f-b546-c636d746963a service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:26:57.066780 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9f638-1f0b-4e4f-b546-c636d746963a service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:26:57.067064 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9f638-1f0b-4e4f-b546-c636d746963a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:26:57.068726 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-87f9f638-1f0b-4e4f-b546-c636d746963a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:26:57.143086 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:26:57.182827 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f9f638-1f0b-4e4f-b546-c636d746963a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:26:57.184493 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f9f638-1f0b-4e4f-b546-c636d746963a service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:26:57.205131 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 542/1086] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:26:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 497 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:27:06.407195 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=67 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:27:07.201172 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=20 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:27:10.438601 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc200b0e-caef-4fe9-a761-17b67cc61530 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:10.439148 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc200b0e-caef-4fe9-a761-17b67cc61530 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:10.439148 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc200b0e-caef-4fe9-a761-17b67cc61530 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:10.440526 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 545/1087] 10.222.0.26 () {54 vars in 918 bytes} [Wed Nov 2 03:27:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:27:10.445550 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c7921-1d5d-418f-92c5-f079a3b84e0e None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:10.445996 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c7921-1d5d-418f-92c5-f079a3b84e0e None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:10.446102 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8c7921-1d5d-418f-92c5-f079a3b84e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:10.522490 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-1f8c7921-1d5d-418f-92c5-f079a3b84e0e None None] MFA Rules not processed for user `9b4b9bd741574ddabadd4b7651ebc885`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:27:10.523748 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-1f8c7921-1d5d-418f-92c5-f079a3b84e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:10.629719 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 543/1088] 10.222.0.26 () {58 vars in 1015 bytes} [Wed Nov 2 03:27:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 03:27:10.643846 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d669f-a3ee-4a43-ae2c-d5add346864b None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:10.644160 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d669f-a3ee-4a43-ae2c-d5add346864b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:10.644460 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d669f-a3ee-4a43-ae2c-d5add346864b None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:10.645777 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 546/1089] 10.222.0.26 () {54 vars in 918 bytes} [Wed Nov 2 03:27:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:27:10.650320 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff27ef4-0357-449a-935e-81f9c593661d None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:10.650624 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff27ef4-0357-449a-935e-81f9c593661d None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:10.650917 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff27ef4-0357-449a-935e-81f9c593661d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:10.722696 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:27:10.725709 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-fff27ef4-0357-449a-935e-81f9c593661d None None] MFA Rules not processed for user `44961b06151c4da382fcd54dd1f1e146`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:27:10.727534 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-fff27ef4-0357-449a-935e-81f9c593661d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:10.842710 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 544/1090] 10.222.0.26 () {58 vars in 1015 bytes} [Wed Nov 2 03:27:10 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 03:27:13.427891 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9805ca0b-a91e-48a8-a303-34638f5a38d5 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:13.428476 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9805ca0b-a91e-48a8-a303-34638f5a38d5 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:13.428476 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9805ca0b-a91e-48a8-a303-34638f5a38d5 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:13.429814 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 547/1091] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 03:27:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:27:13.435619 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fe86ac-1e7b-4979-9a2a-e5c7acbd85ca None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:13.436066 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fe86ac-1e7b-4979-9a2a-e5c7acbd85ca None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:13.436180 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fe86ac-1e7b-4979-9a2a-e5c7acbd85ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:13.502359 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-f7fe86ac-1e7b-4979-9a2a-e5c7acbd85ca None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:27:13.503564 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f7fe86ac-1e7b-4979-9a2a-e5c7acbd85ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:13.562816 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:27:13.605337 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 545/1092] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 03:27:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:27:13.895586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9805ca0b-a91e-48a8-a303-34638f5a38d5 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:27:13.897163 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9805ca0b-a91e-48a8-a303-34638f5a38d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:14.039658 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9805ca0b-a91e-48a8-a303-34638f5a38d5 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:27:14.111048 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:27:14.152730 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d85ed16-ac66-4b8f-9e30-86c05e87d646 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:27:14.153260 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d85ed16-ac66-4b8f-9e30-86c05e87d646 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:14.153534 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d85ed16-ac66-4b8f-9e30-86c05e87d646 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:14.153801 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d85ed16-ac66-4b8f-9e30-86c05e87d646 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:14.155382 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5d85ed16-ac66-4b8f-9e30-86c05e87d646 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:14.242315 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d85ed16-ac66-4b8f-9e30-86c05e87d646 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:27:14.243884 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d85ed16-ac66-4b8f-9e30-86c05e87d646 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:27:14.262360 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 548/1093] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:27:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:27:15.383350 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f4264-2b3e-4505-999b-2ab83b6b0f15 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:15.383925 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f4264-2b3e-4505-999b-2ab83b6b0f15 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:15.383925 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4f4264-2b3e-4505-999b-2ab83b6b0f15 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:15.385814 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 546/1094] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 03:27:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:27:15.390564 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f3186-11f9-4e66-a077-144a9042f6d3 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:15.390994 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f3186-11f9-4e66-a077-144a9042f6d3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:15.391092 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958f3186-11f9-4e66-a077-144a9042f6d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:15.466283 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-958f3186-11f9-4e66-a077-144a9042f6d3 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:27:15.467487 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-958f3186-11f9-4e66-a077-144a9042f6d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:15.494235 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:27:15.564323 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 549/1095] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 03:27:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:27:15.722464 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f4264-2b3e-4505-999b-2ab83b6b0f15 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:27:15.724001 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-da4f4264-2b3e-4505-999b-2ab83b6b0f15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:15.867664 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4f4264-2b3e-4505-999b-2ab83b6b0f15 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:27:15.908916 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:27:15.981276 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da9376-1ad2-4677-ae0e-223c8efe94ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:27:15.981800 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da9376-1ad2-4677-ae0e-223c8efe94ec service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:15.982074 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da9376-1ad2-4677-ae0e-223c8efe94ec service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:15.982341 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da9376-1ad2-4677-ae0e-223c8efe94ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:15.983917 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-b3da9376-1ad2-4677-ae0e-223c8efe94ec service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:16.071427 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3da9376-1ad2-4677-ae0e-223c8efe94ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:27:16.073507 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3da9376-1ad2-4677-ae0e-223c8efe94ec service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:27:16.091524 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 547/1096] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:27:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:27:17.188056 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735bbc4c-baeb-4b88-915d-22c6e0768b75 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:17.188630 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735bbc4c-baeb-4b88-915d-22c6e0768b75 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:17.188630 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735bbc4c-baeb-4b88-915d-22c6e0768b75 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:17.189962 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 550/1097] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 03:27:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:27:17.194687 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb503a9-8e97-454b-a7e6-ad7a31b438a8 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:17.195109 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb503a9-8e97-454b-a7e6-ad7a31b438a8 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:17.195224 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb503a9-8e97-454b-a7e6-ad7a31b438a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:17.251134 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:27:17.274159 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-2bb503a9-8e97-454b-a7e6-ad7a31b438a8 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:27:17.275469 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-2bb503a9-8e97-454b-a7e6-ad7a31b438a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:17.371848 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 548/1098] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 03:27:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:27:17.520547 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735bbc4c-baeb-4b88-915d-22c6e0768b75 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:27:17.522063 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-735bbc4c-baeb-4b88-915d-22c6e0768b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:17.669886 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735bbc4c-baeb-4b88-915d-22c6e0768b75 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:27:17.682031 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:27:17.784282 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e5bccee-f567-4399-bdc6-09758e7949be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:27:17.784804 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5bccee-f567-4399-bdc6-09758e7949be service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:17.785077 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5bccee-f567-4399-bdc6-09758e7949be service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:17.785346 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5bccee-f567-4399-bdc6-09758e7949be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:17.786930 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4e5bccee-f567-4399-bdc6-09758e7949be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:17.874317 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5bccee-f567-4399-bdc6-09758e7949be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:27:17.875894 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e5bccee-f567-4399-bdc6-09758e7949be service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:27:17.893843 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 551/1099] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:27:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:27:18.970059 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1bf549-de51-4baf-bc1c-ca879aba6510 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:18.970581 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1bf549-de51-4baf-bc1c-ca879aba6510 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:18.970677 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1bf549-de51-4baf-bc1c-ca879aba6510 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:18.971989 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 549/1100] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 03:27:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:27:18.976699 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4264b9-3018-4844-b660-4a943b18edf1 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:18.977119 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4264b9-3018-4844-b660-4a943b18edf1 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:18.977244 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4264b9-3018-4844-b660-4a943b18edf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:18.997888 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:27:19.057527 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-9a4264b9-3018-4844-b660-4a943b18edf1 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:27:19.058719 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9a4264b9-3018-4844-b660-4a943b18edf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:19.153954 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 552/1101] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 03:27:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:27:19.305262 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1bf549-de51-4baf-bc1c-ca879aba6510 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:27:19.306711 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-bd1bf549-de51-4baf-bc1c-ca879aba6510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:19.414959 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:27:19.450088 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1bf549-de51-4baf-bc1c-ca879aba6510 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:27:19.562875 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17588f55-5f11-4a80-a70a-4d4df8827162 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:27:19.563410 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17588f55-5f11-4a80-a70a-4d4df8827162 service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:19.563688 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17588f55-5f11-4a80-a70a-4d4df8827162 service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:19.563969 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17588f55-5f11-4a80-a70a-4d4df8827162 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:19.565565 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-17588f55-5f11-4a80-a70a-4d4df8827162 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:19.641173 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:27:19.655133 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17588f55-5f11-4a80-a70a-4d4df8827162 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:27:19.656748 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17588f55-5f11-4a80-a70a-4d4df8827162 service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:27:19.674339 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 550/1102] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:27:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:27:20.755423 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa4470-fda2-4008-abaf-0b559aebe721 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:20.755917 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa4470-fda2-4008-abaf-0b559aebe721 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:20.756062 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fa4470-fda2-4008-abaf-0b559aebe721 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:20.757349 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 553/1103] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 03:27:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:27:20.762112 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32673bfa-6412-4727-8004-364dd0b328df None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:20.762580 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32673bfa-6412-4727-8004-364dd0b328df None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:20.762687 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32673bfa-6412-4727-8004-364dd0b328df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:20.839120 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-32673bfa-6412-4727-8004-364dd0b328df None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:27:20.840270 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-32673bfa-6412-4727-8004-364dd0b328df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:20.936677 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 551/1104] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 03:27:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:27:21.086165 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa4470-fda2-4008-abaf-0b559aebe721 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:27:21.087711 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-91fa4470-fda2-4008-abaf-0b559aebe721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:21.171350 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:27:21.233442 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fa4470-fda2-4008-abaf-0b559aebe721 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:27:21.347767 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0ac21c-1122-4ee9-a4e0-d098ac471494 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:27:21.348400 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ac21c-1122-4ee9-a4e0-d098ac471494 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:21.348673 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ac21c-1122-4ee9-a4e0-d098ac471494 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:21.348938 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0ac21c-1122-4ee9-a4e0-d098ac471494 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:21.350502 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9b0ac21c-1122-4ee9-a4e0-d098ac471494 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:21.398312 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:27:21.439892 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0ac21c-1122-4ee9-a4e0-d098ac471494 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:27:21.441434 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0ac21c-1122-4ee9-a4e0-d098ac471494 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:27:21.459140 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 554/1105] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:27:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:27:22.495813 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84928817-e782-4f41-a55f-fc83cb90abde None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:22.496461 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84928817-e782-4f41-a55f-fc83cb90abde None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:22.496461 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84928817-e782-4f41-a55f-fc83cb90abde None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:22.497772 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 552/1106] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 03:27:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 03:27:22.502789 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6373e088-494e-4cb2-a7fc-02fbed9562f7 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:22.503244 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6373e088-494e-4cb2-a7fc-02fbed9562f7 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:22.503352 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6373e088-494e-4cb2-a7fc-02fbed9562f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:22.588393 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-6373e088-494e-4cb2-a7fc-02fbed9562f7 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:27:22.589604 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6373e088-494e-4cb2-a7fc-02fbed9562f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:22.695216 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 555/1107] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 03:27:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:27:22.849976 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84928817-e782-4f41-a55f-fc83cb90abde None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:27:22.852671 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-84928817-e782-4f41-a55f-fc83cb90abde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:22.902123 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:27:23.001669 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84928817-e782-4f41-a55f-fc83cb90abde None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:27:23.120279 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabd88cd-2d57-4e00-bdfb-0c699ce61972 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:27:23.120884 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd88cd-2d57-4e00-bdfb-0c699ce61972 service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:23.121156 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd88cd-2d57-4e00-bdfb-0c699ce61972 service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:23.121423 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd88cd-2d57-4e00-bdfb-0c699ce61972 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:23.123000 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-fabd88cd-2d57-4e00-bdfb-0c699ce61972 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:23.129968 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:27:23.216864 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabd88cd-2d57-4e00-bdfb-0c699ce61972 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:27:23.218592 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabd88cd-2d57-4e00-bdfb-0c699ce61972 service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:27:23.238536 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 553/1108] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:27:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:27:24.266752 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8271e2-1d7a-41f1-88dd-1cbd207efc24 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:24.267309 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8271e2-1d7a-41f1-88dd-1cbd207efc24 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:24.267309 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8271e2-1d7a-41f1-88dd-1cbd207efc24 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:24.268724 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 556/1109] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 03:27:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:27:24.273713 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c637594-2259-48b8-a2dd-7787cb466d9d None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:24.274146 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c637594-2259-48b8-a2dd-7787cb466d9d None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:24.274252 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c637594-2259-48b8-a2dd-7787cb466d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:24.349829 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-8c637594-2259-48b8-a2dd-7787cb466d9d None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:27:24.351029 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-8c637594-2259-48b8-a2dd-7787cb466d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:24.418504 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:27:24.447713 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 554/1110] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 03:27:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:27:24.596868 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8271e2-1d7a-41f1-88dd-1cbd207efc24 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:27:24.598374 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5e8271e2-1d7a-41f1-88dd-1cbd207efc24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:24.616291 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:27:24.742923 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8271e2-1d7a-41f1-88dd-1cbd207efc24 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:27:24.828365 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:27:24.854719 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19050527-ca50-4ae2-a03e-ef8a012e74c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:27:24.855311 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19050527-ca50-4ae2-a03e-ef8a012e74c8 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:24.855584 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19050527-ca50-4ae2-a03e-ef8a012e74c8 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:24.855848 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19050527-ca50-4ae2-a03e-ef8a012e74c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:24.857450 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-19050527-ca50-4ae2-a03e-ef8a012e74c8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:24.943477 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19050527-ca50-4ae2-a03e-ef8a012e74c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:27:24.945070 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19050527-ca50-4ae2-a03e-ef8a012e74c8 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:27:24.963698 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 557/1111] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:27:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:27:26.003686 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9499194f-c2eb-4159-bf5f-4bef7c84bb26 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:26.004314 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9499194f-c2eb-4159-bf5f-4bef7c84bb26 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:26.004314 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9499194f-c2eb-4159-bf5f-4bef7c84bb26 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:26.005633 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 555/1112] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 03:27:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:27:26.010698 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436111b8-6779-4ebb-8090-a18b476f3d22 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:26.011109 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436111b8-6779-4ebb-8090-a18b476f3d22 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:26.011277 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436111b8-6779-4ebb-8090-a18b476f3d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:26.086449 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-436111b8-6779-4ebb-8090-a18b476f3d22 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:27:26.087662 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-436111b8-6779-4ebb-8090-a18b476f3d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:26.129248 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:27:26.183557 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 558/1113] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 03:27:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:27:26.332932 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9499194f-c2eb-4159-bf5f-4bef7c84bb26 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:27:26.334501 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-9499194f-c2eb-4159-bf5f-4bef7c84bb26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:26.479547 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9499194f-c2eb-4159-bf5f-4bef7c84bb26 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:27:26.533630 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:27:26.592289 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524adffd-e197-4707-816c-c98bf3ef94bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:27:26.592892 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524adffd-e197-4707-816c-c98bf3ef94bb service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:26.593170 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524adffd-e197-4707-816c-c98bf3ef94bb service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:26.593440 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524adffd-e197-4707-816c-c98bf3ef94bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:26.595007 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-524adffd-e197-4707-816c-c98bf3ef94bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:26.683437 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524adffd-e197-4707-816c-c98bf3ef94bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:27:26.685030 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524adffd-e197-4707-816c-c98bf3ef94bb service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:27:26.702928 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 556/1114] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:27:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:27:27.731787 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db7bc20-6eca-465e-85d9-a1775eef5b0a None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:27.732385 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db7bc20-6eca-465e-85d9-a1775eef5b0a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:27.732385 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db7bc20-6eca-465e-85d9-a1775eef5b0a None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:27.733709 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 559/1115] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 03:27:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:27:27.738803 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb93bf-e162-416b-b0dc-e7b63848ddfd None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:27.739196 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb93bf-e162-416b-b0dc-e7b63848ddfd None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:27.739350 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eb93bf-e162-416b-b0dc-e7b63848ddfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:27.813162 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-72eb93bf-e162-416b-b0dc-e7b63848ddfd None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:27:27.814267 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-72eb93bf-e162-416b-b0dc-e7b63848ddfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:27.816277 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:27:27.910995 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 557/1116] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 03:27:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:27:28.059500 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db7bc20-6eca-465e-85d9-a1775eef5b0a None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:27:28.061056 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7db7bc20-6eca-465e-85d9-a1775eef5b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:28.206431 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db7bc20-6eca-465e-85d9-a1775eef5b0a None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:27:28.234073 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:27:28.319048 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41926c35-cfdf-4df9-b54d-725d53a37dae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:27:28.319656 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41926c35-cfdf-4df9-b54d-725d53a37dae service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:28.319979 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41926c35-cfdf-4df9-b54d-725d53a37dae service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:28.320267 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41926c35-cfdf-4df9-b54d-725d53a37dae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:28.321851 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-41926c35-cfdf-4df9-b54d-725d53a37dae service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:28.409015 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41926c35-cfdf-4df9-b54d-725d53a37dae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:27:28.410578 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41926c35-cfdf-4df9-b54d-725d53a37dae service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:27:28.428824 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 560/1117] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:27:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:27:30.471063 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00a2da-91eb-4994-8dce-a1f3421a2850 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:30.471665 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00a2da-91eb-4994-8dce-a1f3421a2850 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:30.471665 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00a2da-91eb-4994-8dce-a1f3421a2850 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:30.472968 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 558/1118] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 03:27:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:27:30.478359 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3a3f07-7c12-4267-8335-ed12779eb157 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:30.478815 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3a3f07-7c12-4267-8335-ed12779eb157 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:30.478939 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3a3f07-7c12-4267-8335-ed12779eb157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:30.520142 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:27:30.546957 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-2a3a3f07-7c12-4267-8335-ed12779eb157 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:27:30.548176 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2a3a3f07-7c12-4267-8335-ed12779eb157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:30.643505 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 561/1119] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 03:27:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:27:30.927068 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab00a2da-91eb-4994-8dce-a1f3421a2850 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:27:30.928747 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ab00a2da-91eb-4994-8dce-a1f3421a2850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:31.031192 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab00a2da-91eb-4994-8dce-a1f3421a2850 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:27:31.063725 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:27:31.105182 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d457d085-8592-49e7-9b0f-3e2f6511b8c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:27:31.105786 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d457d085-8592-49e7-9b0f-3e2f6511b8c3 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:31.106065 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d457d085-8592-49e7-9b0f-3e2f6511b8c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:31.106333 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d457d085-8592-49e7-9b0f-3e2f6511b8c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:31.107916 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d457d085-8592-49e7-9b0f-3e2f6511b8c3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:31.216945 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d457d085-8592-49e7-9b0f-3e2f6511b8c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:27:31.218378 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d457d085-8592-49e7-9b0f-3e2f6511b8c3 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:27:31.237265 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 559/1120] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:27:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:27:33.538708 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ffcd55-591d-4ddd-8cc2-b3dfce276003 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:33.539162 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ffcd55-591d-4ddd-8cc2-b3dfce276003 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:33.539274 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ffcd55-591d-4ddd-8cc2-b3dfce276003 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:33.540786 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 562/1121] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 03:27:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:27:33.545930 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b062b24-450f-4211-8c1f-8f80842727b4 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:33.546382 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b062b24-450f-4211-8c1f-8f80842727b4 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:33.546490 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b062b24-450f-4211-8c1f-8f80842727b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:33.612658 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-7b062b24-450f-4211-8c1f-8f80842727b4 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:27:33.613896 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-7b062b24-450f-4211-8c1f-8f80842727b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:33.627062 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:27:33.710460 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 560/1122] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 03:27:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:27:34.007403 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ffcd55-591d-4ddd-8cc2-b3dfce276003 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:27:34.009013 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-10ffcd55-591d-4ddd-8cc2-b3dfce276003 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:34.110907 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10ffcd55-591d-4ddd-8cc2-b3dfce276003 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:27:34.124089 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:27:34.184992 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a5764d-a70d-4206-ab54-c8bfdf755cda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:27:34.185596 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a5764d-a70d-4206-ab54-c8bfdf755cda service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:34.185877 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a5764d-a70d-4206-ab54-c8bfdf755cda service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:34.186151 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a5764d-a70d-4206-ab54-c8bfdf755cda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:34.187769 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-96a5764d-a70d-4206-ab54-c8bfdf755cda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:34.296645 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a5764d-a70d-4206-ab54-c8bfdf755cda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:27:34.298115 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a5764d-a70d-4206-ab54-c8bfdf755cda service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:27:34.316359 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 563/1123] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:27:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:27:36.749520 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b216d3b-8ed5-44bf-8832-c56ba902c89e None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:36.750107 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b216d3b-8ed5-44bf-8832-c56ba902c89e None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:36.750107 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b216d3b-8ed5-44bf-8832-c56ba902c89e None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:36.751421 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 561/1124] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 03:27:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:27:36.756711 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2dc088-f810-4d96-a0ed-0531c2031008 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:36.757146 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2dc088-f810-4d96-a0ed-0531c2031008 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:36.757243 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b2dc088-f810-4d96-a0ed-0531c2031008 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:36.805284 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:27:36.823386 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-6b2dc088-f810-4d96-a0ed-0531c2031008 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:27:36.824614 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6b2dc088-f810-4d96-a0ed-0531c2031008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:36.919759 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 564/1125] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 03:27:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:27:37.206923 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b216d3b-8ed5-44bf-8832-c56ba902c89e None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:27:37.208527 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-5b216d3b-8ed5-44bf-8832-c56ba902c89e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:37.313921 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b216d3b-8ed5-44bf-8832-c56ba902c89e None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:27:37.355384 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:27:37.389280 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204570bc-2180-4374-b504-e8b1b3a6b10d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:27:37.389884 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204570bc-2180-4374-b504-e8b1b3a6b10d service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:37.390162 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204570bc-2180-4374-b504-e8b1b3a6b10d service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:37.390429 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204570bc-2180-4374-b504-e8b1b3a6b10d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:37.392029 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-204570bc-2180-4374-b504-e8b1b3a6b10d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:37.500137 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204570bc-2180-4374-b504-e8b1b3a6b10d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:27:37.501555 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204570bc-2180-4374-b504-e8b1b3a6b10d service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:27:37.518960 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 562/1126] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:27:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:27:39.761458 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a6389-badb-4668-ba9b-e6ad47dad314 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:39.762032 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a6389-badb-4668-ba9b-e6ad47dad314 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:39.762032 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077a6389-badb-4668-ba9b-e6ad47dad314 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:39.763387 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 565/1127] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 03:27:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:27:39.768783 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec714e8-55b9-4244-8cfd-024b8a220fb2 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:39.769215 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec714e8-55b9-4244-8cfd-024b8a220fb2 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:39.769498 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec714e8-55b9-4244-8cfd-024b8a220fb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:39.835059 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-1ec714e8-55b9-4244-8cfd-024b8a220fb2 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:27:39.836215 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-1ec714e8-55b9-4244-8cfd-024b8a220fb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:39.857408 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:27:39.934070 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 563/1128] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 03:27:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:27:40.214371 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077a6389-badb-4668-ba9b-e6ad47dad314 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:27:40.215931 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-077a6389-badb-4668-ba9b-e6ad47dad314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:40.318276 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077a6389-badb-4668-ba9b-e6ad47dad314 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:27:40.338797 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:27:40.390757 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0248cb44-dc08-478b-b537-b4266c7df926 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:27:40.391344 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0248cb44-dc08-478b-b537-b4266c7df926 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:40.391622 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0248cb44-dc08-478b-b537-b4266c7df926 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:40.391894 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0248cb44-dc08-478b-b537-b4266c7df926 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:40.393505 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0248cb44-dc08-478b-b537-b4266c7df926 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:40.500754 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0248cb44-dc08-478b-b537-b4266c7df926 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:27:40.502213 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0248cb44-dc08-478b-b537-b4266c7df926 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:27:40.520086 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 566/1129] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:27:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:27:42.707944 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef4052f-64ef-4f68-b466-dc49bf11975d None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:42.708522 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef4052f-64ef-4f68-b466-dc49bf11975d None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:42.708522 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef4052f-64ef-4f68-b466-dc49bf11975d None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:42.709886 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 564/1130] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 03:27:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:27:42.715443 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a97ab92-4bb0-4eb3-93ba-e0e4f5389b04 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:42.715863 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a97ab92-4bb0-4eb3-93ba-e0e4f5389b04 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:42.716114 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a97ab92-4bb0-4eb3-93ba-e0e4f5389b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:42.782730 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:27:42.785930 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-6a97ab92-4bb0-4eb3-93ba-e0e4f5389b04 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:27:42.787820 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6a97ab92-4bb0-4eb3-93ba-e0e4f5389b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:42.884271 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 567/1131] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 03:27:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:27:43.167673 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef4052f-64ef-4f68-b466-dc49bf11975d None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:27:43.169571 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-1ef4052f-64ef-4f68-b466-dc49bf11975d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:43.273739 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef4052f-64ef-4f68-b466-dc49bf11975d None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:27:43.322858 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:27:43.347184 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adcd08c-952b-45e7-b771-e6102745c2cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:27:43.347785 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcd08c-952b-45e7-b771-e6102745c2cc service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:43.348112 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcd08c-952b-45e7-b771-e6102745c2cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:43.348365 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adcd08c-952b-45e7-b771-e6102745c2cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:43.349931 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-7adcd08c-952b-45e7-b771-e6102745c2cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:43.458790 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adcd08c-952b-45e7-b771-e6102745c2cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:27:43.460200 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adcd08c-952b-45e7-b771-e6102745c2cc service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:27:43.479571 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 565/1132] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:27:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:27:45.658498 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfe7a82-f08e-43a2-982d-be0d95509a96 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:45.659051 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfe7a82-f08e-43a2-982d-be0d95509a96 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:45.659051 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbfe7a82-f08e-43a2-982d-be0d95509a96 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:45.660413 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 568/1133] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 03:27:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:27:45.666283 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a3b8d7-c276-48df-8387-e6227db83eb5 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:45.666703 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a3b8d7-c276-48df-8387-e6227db83eb5 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:45.666858 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8a3b8d7-c276-48df-8387-e6227db83eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:45.732762 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-f8a3b8d7-c276-48df-8387-e6227db83eb5 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:27:45.733966 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f8a3b8d7-c276-48df-8387-e6227db83eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:45.762785 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:27:45.829502 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 566/1134] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 03:27:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:27:46.113520 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfe7a82-f08e-43a2-982d-be0d95509a96 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:27:46.115048 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-cbfe7a82-f08e-43a2-982d-be0d95509a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:46.222074 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbfe7a82-f08e-43a2-982d-be0d95509a96 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:27:46.251691 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:27:46.296741 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b672e622-a972-4759-84c9-06da9adccbe9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:27:46.297344 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672e622-a972-4759-84c9-06da9adccbe9 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:46.297617 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672e622-a972-4759-84c9-06da9adccbe9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:46.297885 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b672e622-a972-4759-84c9-06da9adccbe9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:46.299453 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b672e622-a972-4759-84c9-06da9adccbe9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:46.407484 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b672e622-a972-4759-84c9-06da9adccbe9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:27:46.408865 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b672e622-a972-4759-84c9-06da9adccbe9 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:27:46.427330 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 569/1135] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:27:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:27:48.605519 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d01d426-0567-4aa3-ba20-f7e5d57c2b1d None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:48.606051 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d01d426-0567-4aa3-ba20-f7e5d57c2b1d None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:48.606165 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d01d426-0567-4aa3-ba20-f7e5d57c2b1d None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:48.607462 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 567/1136] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 03:27:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:27:48.613006 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ceebba-82d1-4787-9df6-c0423e487706 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:48.613425 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ceebba-82d1-4787-9df6-c0423e487706 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:48.613560 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ceebba-82d1-4787-9df6-c0423e487706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:48.678585 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-97ceebba-82d1-4787-9df6-c0423e487706 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:27:48.679791 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-97ceebba-82d1-4787-9df6-c0423e487706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:48.688296 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:27:48.775596 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 570/1137] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 03:27:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:27:49.058940 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d01d426-0567-4aa3-ba20-f7e5d57c2b1d None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:27:49.060533 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-1d01d426-0567-4aa3-ba20-f7e5d57c2b1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:49.163663 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d01d426-0567-4aa3-ba20-f7e5d57c2b1d None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:27:49.219172 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:27:49.237246 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da0fee0-0a9f-4405-86c3-5fd7b8bb8839 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:27:49.237842 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da0fee0-0a9f-4405-86c3-5fd7b8bb8839 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:49.238120 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da0fee0-0a9f-4405-86c3-5fd7b8bb8839 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:49.238420 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da0fee0-0a9f-4405-86c3-5fd7b8bb8839 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:49.240030 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-3da0fee0-0a9f-4405-86c3-5fd7b8bb8839 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:49.349811 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da0fee0-0a9f-4405-86c3-5fd7b8bb8839 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:27:49.351156 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da0fee0-0a9f-4405-86c3-5fd7b8bb8839 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:27:49.369437 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 568/1138] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:27:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:27:51.545909 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225d537-01a1-490d-a001-b46d8ff5c49f None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:51.546430 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225d537-01a1-490d-a001-b46d8ff5c49f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:51.546430 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4225d537-01a1-490d-a001-b46d8ff5c49f None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:51.547720 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 571/1139] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 03:27:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:27:51.553036 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359916a2-38bf-489f-8a18-b7acd5b6d41d None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:51.553500 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359916a2-38bf-489f-8a18-b7acd5b6d41d None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:51.553608 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359916a2-38bf-489f-8a18-b7acd5b6d41d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:51.620591 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-359916a2-38bf-489f-8a18-b7acd5b6d41d None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:27:51.621807 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-359916a2-38bf-489f-8a18-b7acd5b6d41d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:51.659649 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:27:51.718290 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 569/1140] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 03:27:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:27:51.999429 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4225d537-01a1-490d-a001-b46d8ff5c49f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:27:52.001010 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4225d537-01a1-490d-a001-b46d8ff5c49f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:52.102315 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4225d537-01a1-490d-a001-b46d8ff5c49f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:27:52.139627 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:27:52.174391 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3901c494-8212-4cc1-a19a-b07bfd2fd60a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:27:52.174989 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901c494-8212-4cc1-a19a-b07bfd2fd60a service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:52.175262 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901c494-8212-4cc1-a19a-b07bfd2fd60a service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:52.175526 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3901c494-8212-4cc1-a19a-b07bfd2fd60a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:52.177128 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3901c494-8212-4cc1-a19a-b07bfd2fd60a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:52.285561 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3901c494-8212-4cc1-a19a-b07bfd2fd60a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:27:52.287021 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3901c494-8212-4cc1-a19a-b07bfd2fd60a service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:27:52.304736 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 572/1141] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:27:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:27:54.474390 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9d48cf-fbce-455d-b3f2-85db1b78ee57 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:54.474948 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9d48cf-fbce-455d-b3f2-85db1b78ee57 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:54.474948 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9d48cf-fbce-455d-b3f2-85db1b78ee57 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:54.476494 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 570/1142] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 03:27:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:27:54.482873 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f7422e-c376-4c7e-8e21-9085181574a8 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:54.483351 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f7422e-c376-4c7e-8e21-9085181574a8 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:54.483462 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f7422e-c376-4c7e-8e21-9085181574a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:54.549431 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-87f7422e-c376-4c7e-8e21-9085181574a8 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:27:54.550628 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-87f7422e-c376-4c7e-8e21-9085181574a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:54.568943 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:27:54.647098 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 573/1143] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 03:27:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:27:54.933508 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9d48cf-fbce-455d-b3f2-85db1b78ee57 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:27:54.935056 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-1e9d48cf-fbce-455d-b3f2-85db1b78ee57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:55.035127 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9d48cf-fbce-455d-b3f2-85db1b78ee57 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:27:55.099992 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:27:55.107257 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7413bb23-b394-48cc-8834-9ce0a12796a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:27:55.107856 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7413bb23-b394-48cc-8834-9ce0a12796a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:55.108158 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7413bb23-b394-48cc-8834-9ce0a12796a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:55.108433 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7413bb23-b394-48cc-8834-9ce0a12796a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:55.109998 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-7413bb23-b394-48cc-8834-9ce0a12796a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:55.216794 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7413bb23-b394-48cc-8834-9ce0a12796a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:27:55.218232 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7413bb23-b394-48cc-8834-9ce0a12796a7 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:27:55.236777 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 571/1144] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:27:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:27:57.389170 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7239dfc7-7237-43d0-bd5a-91fc5b182169 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:57.389739 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7239dfc7-7237-43d0-bd5a-91fc5b182169 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:57.389739 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7239dfc7-7237-43d0-bd5a-91fc5b182169 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:57.391047 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 574/1145] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 03:27:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:27:57.396256 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5800fa9-7b9d-4b38-93d0-0f494e8aea5e None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:57.396696 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5800fa9-7b9d-4b38-93d0-0f494e8aea5e None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:57.396806 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5800fa9-7b9d-4b38-93d0-0f494e8aea5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:57.462002 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-c5800fa9-7b9d-4b38-93d0-0f494e8aea5e None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:27:57.463204 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-c5800fa9-7b9d-4b38-93d0-0f494e8aea5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:57.512802 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:27:57.562866 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 572/1146] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 03:27:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:27:57.841992 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7239dfc7-7237-43d0-bd5a-91fc5b182169 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:27:57.843518 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7239dfc7-7237-43d0-bd5a-91fc5b182169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:57.944911 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7239dfc7-7237-43d0-bd5a-91fc5b182169 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:27:57.992586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:27:58.018243 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de560a62-4087-4cbe-8aa3-13886aebb825 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:27:58.018848 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de560a62-4087-4cbe-8aa3-13886aebb825 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:27:58.019124 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de560a62-4087-4cbe-8aa3-13886aebb825 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:27:58.019391 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de560a62-4087-4cbe-8aa3-13886aebb825 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:27:58.021011 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-de560a62-4087-4cbe-8aa3-13886aebb825 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:27:58.126870 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de560a62-4087-4cbe-8aa3-13886aebb825 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:27:58.128249 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de560a62-4087-4cbe-8aa3-13886aebb825 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:27:58.146130 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 575/1147] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:27:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:28:07.561318 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=22 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:28:08.143898 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=66 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:28:17.874030 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25126216-7492-48e4-b968-3692e26c53e2 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:28:17.874572 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25126216-7492-48e4-b968-3692e26c53e2 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:28:17.874572 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25126216-7492-48e4-b968-3692e26c53e2 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:28:17.875837 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 573/1148] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:28:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:28:17.881135 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb38b24-7df7-43c5-934b-bff7b10c5b2b None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:28:17.881534 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb38b24-7df7-43c5-934b-bff7b10c5b2b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:28:17.881682 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb38b24-7df7-43c5-934b-bff7b10c5b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:28:17.947176 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-7fb38b24-7df7-43c5-934b-bff7b10c5b2b None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:28:17.948403 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7fb38b24-7df7-43c5-934b-bff7b10c5b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:28:18.046881 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 576/1149] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:28:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:28:18.347657 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc9989-2585-409d-8a4d-7cb77584aafa None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:28:18.347943 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc9989-2585-409d-8a4d-7cb77584aafa None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:28:18.348246 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cc9989-2585-409d-8a4d-7cb77584aafa None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:28:18.349577 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 574/1150] 10.222.0.26 () {54 vars in 962 bytes} [Wed Nov 2 03:28:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:28:18.354643 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ea236-346d-4158-b769-27f72ce215ae None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:28:18.355412 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ea236-346d-4158-b769-27f72ce215ae None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:28:18.355691 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ea236-346d-4158-b769-27f72ce215ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:28:18.430949 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-416ea236-346d-4158-b769-27f72ce215ae None None] MFA Rules not processed for user `50aab624ee9c4651a7cd42124bcc465f`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:28:18.432195 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-416ea236-346d-4158-b769-27f72ce215ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:28:18.452894 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:28:18.507507 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 577/1151] 10.222.0.26 () {58 vars in 1059 bytes} [Wed Nov 2 03:28:18 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:28:18.515426 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cc9989-2585-409d-8a4d-7cb77584aafa None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:28:18.516973 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e0cc9989-2585-409d-8a4d-7cb77584aafa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:28:18.621266 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cc9989-2585-409d-8a4d-7cb77584aafa None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:28:18.694137 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5504d5-ce60-46be-956d-e05b451fa816 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50aab624ee9c4651a7cd42124bcc465f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:28:18.694739 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5504d5-ce60-46be-956d-e05b451fa816 service glance] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:28:18.695019 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5504d5-ce60-46be-956d-e05b451fa816 service glance] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:28:18.695288 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5504d5-ce60-46be-956d-e05b451fa816 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:28:18.696937 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ee5504d5-ce60-46be-956d-e05b451fa816 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:28:18.751220 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:28:18.806501 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5504d5-ce60-46be-956d-e05b451fa816 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:28:18.807984 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5504d5-ce60-46be-956d-e05b451fa816 service glance] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:28:18.826710 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 575/1152] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:28:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:28:21.065994 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47330de-94c0-4790-aaef-d83f4eae6f9d None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:28:21.066526 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47330de-94c0-4790-aaef-d83f4eae6f9d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:28:21.066526 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47330de-94c0-4790-aaef-d83f4eae6f9d None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:28:21.067883 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 578/1153] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:28:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:28:21.073255 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590917e1-a805-414b-a9b6-2be366e15a95 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:28:21.073678 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590917e1-a805-414b-a9b6-2be366e15a95 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:28:21.074427 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590917e1-a805-414b-a9b6-2be366e15a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:28:21.139597 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-590917e1-a805-414b-a9b6-2be366e15a95 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:28:21.140823 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-590917e1-a805-414b-a9b6-2be366e15a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:28:21.236547 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 576/1154] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:28:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:28:21.508731 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47330de-94c0-4790-aaef-d83f4eae6f9d None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:28:21.510191 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e47330de-94c0-4790-aaef-d83f4eae6f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:28:21.635258 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47330de-94c0-4790-aaef-d83f4eae6f9d None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:28:21.685694 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:28:21.734161 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ca3f0c-bed5-4e4f-9396-a3f15f0529c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:28:21.734767 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca3f0c-bed5-4e4f-9396-a3f15f0529c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:28:21.735041 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca3f0c-bed5-4e4f-9396-a3f15f0529c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:28:21.735304 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ca3f0c-bed5-4e4f-9396-a3f15f0529c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:28:21.735933 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ca3f0c-bed5-4e4f-9396-a3f15f0529c3 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:28:21.737308 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: WARNING py.warnings [None req-b0ca3f0c-bed5-4e4f-9396-a3f15f0529c3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 03:28:21.737308 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: warnings.warn(msg) Nov 02 03:28:21.737308 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: Nov 02 03:28:21.738426 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ca3f0c-bed5-4e4f-9396-a3f15f0529c3 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:28:21.757713 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 579/1155] 10.222.0.26 () {56 vars in 1108 bytes} [Wed Nov 2 03:28:21 2022] GET /identity/v3/projects => generated 2553 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:28:23.922160 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab38473d-f9a6-4b26-9d90-09dd93000088 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:28:23.922648 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab38473d-f9a6-4b26-9d90-09dd93000088 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:28:23.922836 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab38473d-f9a6-4b26-9d90-09dd93000088 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:28:23.924143 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 577/1156] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:28:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:28:23.929396 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bbdc6c-54a5-4f80-9bf7-cdfea06340d3 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:28:23.929819 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bbdc6c-54a5-4f80-9bf7-cdfea06340d3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:28:23.929949 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bbdc6c-54a5-4f80-9bf7-cdfea06340d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:28:23.997059 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-87bbdc6c-54a5-4f80-9bf7-cdfea06340d3 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:28:23.998249 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-87bbdc6c-54a5-4f80-9bf7-cdfea06340d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:28:24.098077 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 580/1157] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:28:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 03:28:24.391372 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab38473d-f9a6-4b26-9d90-09dd93000088 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:28:24.392955 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ab38473d-f9a6-4b26-9d90-09dd93000088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:28:24.402600 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:28:24.552985 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab38473d-f9a6-4b26-9d90-09dd93000088 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:28:24.634351 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:28:24.667089 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6bff59-b3ca-4bb9-ba21-839a53acb539 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:28:24.668336 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6bff59-b3ca-4bb9-ba21-839a53acb539 service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:28:24.668626 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6bff59-b3ca-4bb9-ba21-839a53acb539 service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:28:24.668901 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6bff59-b3ca-4bb9-ba21-839a53acb539 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:28:24.670508 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-4e6bff59-b3ca-4bb9-ba21-839a53acb539 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:28:24.760663 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6bff59-b3ca-4bb9-ba21-839a53acb539 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:28:24.762261 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6bff59-b3ca-4bb9-ba21-839a53acb539 service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:28:24.780683 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 578/1158] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:28:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:28:27.103514 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7091b-a2f7-4f80-92c6-886aa6be7830 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:28:27.104103 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7091b-a2f7-4f80-92c6-886aa6be7830 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:28:27.104103 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb7091b-a2f7-4f80-92c6-886aa6be7830 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:28:27.105979 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 581/1159] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:28:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:28:27.111453 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c50b49e-8f89-424e-9dc6-ed8b8930bf8e None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:28:27.111859 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c50b49e-8f89-424e-9dc6-ed8b8930bf8e None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:28:27.112020 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c50b49e-8f89-424e-9dc6-ed8b8930bf8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:28:27.177854 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-9c50b49e-8f89-424e-9dc6-ed8b8930bf8e None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:28:27.179055 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-9c50b49e-8f89-424e-9dc6-ed8b8930bf8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:28:27.225990 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:28:27.277364 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 579/1160] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:28:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:28:27.561927 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7091b-a2f7-4f80-92c6-886aa6be7830 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:28:27.563485 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ceb7091b-a2f7-4f80-92c6-886aa6be7830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:28:27.586667 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:28:27.665635 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb7091b-a2f7-4f80-92c6-886aa6be7830 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:28:27.739280 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba3a9a3-08ee-46cf-b93d-22d56763ce4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:28:27.739887 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba3a9a3-08ee-46cf-b93d-22d56763ce4a service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:28:27.740198 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba3a9a3-08ee-46cf-b93d-22d56763ce4a service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:28:27.740471 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba3a9a3-08ee-46cf-b93d-22d56763ce4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:28:27.743109 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5ba3a9a3-08ee-46cf-b93d-22d56763ce4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:28:27.817199 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:28:27.852457 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba3a9a3-08ee-46cf-b93d-22d56763ce4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:28:27.853873 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba3a9a3-08ee-46cf-b93d-22d56763ce4a service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:28:27.872138 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 582/1161] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:28:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:28:30.055547 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910826af-a213-45eb-86f7-c293bba946ae None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:28:30.056130 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910826af-a213-45eb-86f7-c293bba946ae None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:28:30.056130 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910826af-a213-45eb-86f7-c293bba946ae None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:28:30.057474 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 580/1162] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:28:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:28:30.063289 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc5b229-dd17-49de-b81b-5d5f569eb0b9 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:28:30.063715 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc5b229-dd17-49de-b81b-5d5f569eb0b9 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:28:30.063855 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc5b229-dd17-49de-b81b-5d5f569eb0b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:28:30.129195 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-fbc5b229-dd17-49de-b81b-5d5f569eb0b9 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:28:30.130398 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-fbc5b229-dd17-49de-b81b-5d5f569eb0b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:28:30.223342 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 583/1163] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:28:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:28:30.510082 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910826af-a213-45eb-86f7-c293bba946ae None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:28:30.512301 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-910826af-a213-45eb-86f7-c293bba946ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:28:30.614782 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910826af-a213-45eb-86f7-c293bba946ae None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:28:30.687455 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:28:30.688952 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7c2009-6c86-4431-9c75-2331184409e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:28:30.689558 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c2009-6c86-4431-9c75-2331184409e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:28:30.690018 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c2009-6c86-4431-9c75-2331184409e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:28:30.690475 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7c2009-6c86-4431-9c75-2331184409e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:28:30.692324 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-5f7c2009-6c86-4431-9c75-2331184409e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:28:30.799808 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7c2009-6c86-4431-9c75-2331184409e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:28:30.801189 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7c2009-6c86-4431-9c75-2331184409e9 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:28:30.818808 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 581/1164] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:28:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:28:33.116950 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5f552-31a7-4911-b3a9-a689d170459c None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:28:33.117511 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5f552-31a7-4911-b3a9-a689d170459c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:28:33.117511 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5f552-31a7-4911-b3a9-a689d170459c None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:28:33.118830 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 584/1165] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:28:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:28:33.124775 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef69c0fc-6a03-4b9f-ae29-ed737bf358c9 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:28:33.125161 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef69c0fc-6a03-4b9f-ae29-ed737bf358c9 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:28:33.125323 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef69c0fc-6a03-4b9f-ae29-ed737bf358c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:28:33.190809 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-ef69c0fc-6a03-4b9f-ae29-ed737bf358c9 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:28:33.192036 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ef69c0fc-6a03-4b9f-ae29-ed737bf358c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:28:33.244398 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:28:33.289688 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 582/1166] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:28:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:28:33.580450 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df5f552-31a7-4911-b3a9-a689d170459c None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:28:33.581982 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0df5f552-31a7-4911-b3a9-a689d170459c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:28:33.618872 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:28:33.686455 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df5f552-31a7-4911-b3a9-a689d170459c None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:28:33.758864 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5068125d-4d48-40dc-b56f-23e64478d1e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:28:33.759469 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5068125d-4d48-40dc-b56f-23e64478d1e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:28:33.759750 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5068125d-4d48-40dc-b56f-23e64478d1e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:28:33.760069 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5068125d-4d48-40dc-b56f-23e64478d1e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:28:33.761663 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5068125d-4d48-40dc-b56f-23e64478d1e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:28:33.842411 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:28:33.870640 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5068125d-4d48-40dc-b56f-23e64478d1e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:28:33.872122 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5068125d-4d48-40dc-b56f-23e64478d1e7 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:28:33.890485 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 585/1167] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:28:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:28:36.169960 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e2b00a-96d2-4f42-96fe-b9c7c377c509 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:28:36.170489 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e2b00a-96d2-4f42-96fe-b9c7c377c509 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:28:36.170489 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e2b00a-96d2-4f42-96fe-b9c7c377c509 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:28:36.171758 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 583/1168] 10.222.0.26 () {54 vars in 927 bytes} [Wed Nov 2 03:28:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:28:36.177054 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2636a402-3a88-40c1-b3b5-1880bee8a64d None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:28:36.177484 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2636a402-3a88-40c1-b3b5-1880bee8a64d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:28:36.177605 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2636a402-3a88-40c1-b3b5-1880bee8a64d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:28:36.242511 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-2636a402-3a88-40c1-b3b5-1880bee8a64d None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:28:36.243780 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2636a402-3a88-40c1-b3b5-1880bee8a64d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:28:36.339238 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 586/1169] 10.222.0.26 () {58 vars in 1024 bytes} [Wed Nov 2 03:28:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:28:36.630272 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e2b00a-96d2-4f42-96fe-b9c7c377c509 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:28:36.631816 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-39e2b00a-96d2-4f42-96fe-b9c7c377c509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:28:36.734460 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e2b00a-96d2-4f42-96fe-b9c7c377c509 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:28:36.806194 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c54877-3c37-487f-bbe2-c7f2656b64be service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:28:36.806726 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c54877-3c37-487f-bbe2-c7f2656b64be service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:28:36.807005 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c54877-3c37-487f-bbe2-c7f2656b64be service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:28:36.807279 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c54877-3c37-487f-bbe2-c7f2656b64be service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:28:36.809412 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e5c54877-3c37-487f-bbe2-c7f2656b64be service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:28:36.818593 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:28:36.917204 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c54877-3c37-487f-bbe2-c7f2656b64be service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:28:36.918620 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c54877-3c37-487f-bbe2-c7f2656b64be service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:28:36.936335 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 584/1170] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:28:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:28:46.337749 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=82 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:28:46.934115 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=50 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:29:56.244607 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b89a55-44b9-40d4-81e0-e85133949b0f None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:29:56.245182 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b89a55-44b9-40d4-81e0-e85133949b0f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:29:56.245182 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b89a55-44b9-40d4-81e0-e85133949b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:29:56.319510 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-b3b89a55-44b9-40d4-81e0-e85133949b0f None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:29:56.320734 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b3b89a55-44b9-40d4-81e0-e85133949b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:29:56.416556 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 587/1171] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:29:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:29:56.424924 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910077c-cfdf-49a1-bf50-d9f725e8869c None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:29:56.425409 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910077c-cfdf-49a1-bf50-d9f725e8869c None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:29:56.425511 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4910077c-cfdf-49a1-bf50-d9f725e8869c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:29:56.501345 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-4910077c-cfdf-49a1-bf50-d9f725e8869c None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:29:56.502460 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-4910077c-cfdf-49a1-bf50-d9f725e8869c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:29:56.600741 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 585/1172] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:29:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:29:56.605378 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b89a55-44b9-40d4-81e0-e85133949b0f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:29:56.606620 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b3b89a55-44b9-40d4-81e0-e85133949b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:29:56.618834 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b89a55-44b9-40d4-81e0-e85133949b0f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:29:56.681137 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:29:56.715366 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19249fb0-ddbd-4888-9968-17415018f1e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:29:56.715995 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19249fb0-ddbd-4888-9968-17415018f1e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:29:56.716276 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19249fb0-ddbd-4888-9968-17415018f1e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:29:56.717055 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19249fb0-ddbd-4888-9968-17415018f1e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:29:56.717686 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19249fb0-ddbd-4888-9968-17415018f1e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:29:56.717984 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19249fb0-ddbd-4888-9968-17415018f1e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:29:56.718985 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: WARNING py.warnings [None req-19249fb0-ddbd-4888-9968-17415018f1e2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Nov 02 03:29:56.718985 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: warnings.warn(msg) Nov 02 03:29:56.718985 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: Nov 02 03:29:56.719853 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19249fb0-ddbd-4888-9968-17415018f1e2 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:29:56.730877 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 588/1173] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 03:29:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:29:56.740818 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4910077c-cfdf-49a1-bf50-d9f725e8869c None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:29:56.750436 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4910077c-cfdf-49a1-bf50-d9f725e8869c None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:29:56.812253 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:29:56.845973 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b669aa-af08-4607-9655-c7e4d08be5d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:29:56.846576 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b669aa-af08-4607-9655-c7e4d08be5d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:29:56.846851 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b669aa-af08-4607-9655-c7e4d08be5d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:29:56.847118 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b669aa-af08-4607-9655-c7e4d08be5d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:29:56.847750 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b669aa-af08-4607-9655-c7e4d08be5d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:29:56.848886 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: WARNING py.warnings [None req-f2b669aa-af08-4607-9655-c7e4d08be5d0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 03:29:56.848886 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: warnings.warn(msg) Nov 02 03:29:56.848886 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: Nov 02 03:29:56.850077 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b669aa-af08-4607-9655-c7e4d08be5d0 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:29:56.904505 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 586/1174] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:29:56 2022] POST /identity/v3/projects => generated 377 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:29:56.908741 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19249fb0-ddbd-4888-9968-17415018f1e2 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:29:56.910257 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-19249fb0-ddbd-4888-9968-17415018f1e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:29:56.943027 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19249fb0-ddbd-4888-9968-17415018f1e2 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:29:57.038533 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c53261-ae11-4e29-ab9e-1ae9a0b50c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:29:57.039142 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c53261-ae11-4e29-ab9e-1ae9a0b50c43 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:29:57.039421 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c53261-ae11-4e29-ab9e-1ae9a0b50c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:29:57.039692 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c53261-ae11-4e29-ab9e-1ae9a0b50c43 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:29:57.040364 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c53261-ae11-4e29-ab9e-1ae9a0b50c43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:29:57.041374 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: WARNING py.warnings [None req-84c53261-ae11-4e29-ab9e-1ae9a0b50c43 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 03:29:57.041374 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: warnings.warn(msg) Nov 02 03:29:57.041374 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: Nov 02 03:29:57.042576 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c53261-ae11-4e29-ab9e-1ae9a0b50c43 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:29:57.084445 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 589/1175] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:29:56 2022] POST /identity/v3/users => generated 289 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:29:57.088687 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b669aa-af08-4607-9655-c7e4d08be5d0 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:29:57.090187 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f2b669aa-af08-4607-9655-c7e4d08be5d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:29:57.151109 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b669aa-af08-4607-9655-c7e4d08be5d0 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:29:57.234303 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:29:57.245473 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f3f509-16f7-44f2-844d-117b6ab5604e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:29:57.246079 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f3f509-16f7-44f2-844d-117b6ab5604e admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:29:57.246354 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f3f509-16f7-44f2-844d-117b6ab5604e admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:29:57.246624 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f3f509-16f7-44f2-844d-117b6ab5604e admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:29:57.247293 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f3f509-16f7-44f2-844d-117b6ab5604e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:29:57.249315 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f3f509-16f7-44f2-844d-117b6ab5604e admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:29:57.264835 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 587/1176] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 03:29:57.268673 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c53261-ae11-4e29-ab9e-1ae9a0b50c43 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:29:57.270222 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-84c53261-ae11-4e29-ab9e-1ae9a0b50c43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:29:57.282213 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c53261-ae11-4e29-ab9e-1ae9a0b50c43 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:29:57.306693 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:29:57.376203 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1613d8b4-2b2a-4327-8d65-30290f9b7365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:29:57.376713 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1613d8b4-2b2a-4327-8d65-30290f9b7365 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:29:57.376989 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1613d8b4-2b2a-4327-8d65-30290f9b7365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:29:57.377253 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1613d8b4-2b2a-4327-8d65-30290f9b7365 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:29:57.377878 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1613d8b4-2b2a-4327-8d65-30290f9b7365 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:29:57.379485 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1613d8b4-2b2a-4327-8d65-30290f9b7365 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:29:57.392095 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 590/1177] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:29:57 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:29:57.396404 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f3f509-16f7-44f2-844d-117b6ab5604e admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:29:57.397963 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-10f3f509-16f7-44f2-844d-117b6ab5604e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:29:57.431485 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f3f509-16f7-44f2-844d-117b6ab5604e admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:29:57.525558 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6f3f12-6c6c-43bf-8fd3-08869732144e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:29:57.526221 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f3f12-6c6c-43bf-8fd3-08869732144e admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:29:57.526500 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f3f12-6c6c-43bf-8fd3-08869732144e admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:29:57.526778 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6f3f12-6c6c-43bf-8fd3-08869732144e admin admin] PATH_INFO: `/v3/projects/9ceb9ab23fd84afc90fa0f39131238a0/users/f693d67a771f48e3b459cde924d626cf/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:29:57.549258 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6f3f12-6c6c-43bf-8fd3-08869732144e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9ceb9ab23fd84afc90fa0f39131238a0, user_id=f693d67a771f48e3b459cde924d626cf, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:29:57.551973 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6f3f12-6c6c-43bf-8fd3-08869732144e admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:29:57.576736 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 588/1178] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:29:57 2022] PUT /identity/v3/projects/9ceb9ab23fd84afc90fa0f39131238a0/users/f693d67a771f48e3b459cde924d626cf/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:29:57.587554 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b33cee-9d59-4cd0-9df9-d536da565d3f None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:29:57.587869 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b33cee-9d59-4cd0-9df9-d536da565d3f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:29:57.588186 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b33cee-9d59-4cd0-9df9-d536da565d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:29:57.643408 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-09b33cee-9d59-4cd0-9df9-d536da565d3f None None] MFA Rules not processed for user `f693d67a771f48e3b459cde924d626cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:29:57.644567 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-09b33cee-9d59-4cd0-9df9-d536da565d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:29:57.721960 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 591/1179] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:29:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:29:57.735518 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca293a-2671-4c67-9ab9-65fb0711475a None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:29:57.735806 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca293a-2671-4c67-9ab9-65fb0711475a None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:29:57.736126 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdca293a-2671-4c67-9ab9-65fb0711475a None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:29:57.737500 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 589/1180] 10.222.0.26 () {54 vars in 963 bytes} [Wed Nov 2 03:29:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:29:57.742472 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb299a2-966a-4017-86eb-d69e51d246a5 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:29:57.742763 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb299a2-966a-4017-86eb-d69e51d246a5 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:29:57.743059 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb299a2-966a-4017-86eb-d69e51d246a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:29:57.774428 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:29:57.819174 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-9bb299a2-966a-4017-86eb-d69e51d246a5 None None] MFA Rules not processed for user `df541db335eb42dc8b36cc858fa7c22f`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:29:57.820406 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9bb299a2-966a-4017-86eb-d69e51d246a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:29:57.894615 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 592/1181] 10.222.0.26 () {58 vars in 1060 bytes} [Wed Nov 2 03:29:57 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:29:57.902361 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca293a-2671-4c67-9ab9-65fb0711475a None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:29:57.903868 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-cdca293a-2671-4c67-9ab9-65fb0711475a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:29:57.954566 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:29:58.009922 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdca293a-2671-4c67-9ab9-65fb0711475a None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:29:58.081378 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aaa45c-ac6f-469d-9eaa-1471ac1539e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:29:58.081977 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aaa45c-ac6f-469d-9eaa-1471ac1539e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:29:58.082253 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aaa45c-ac6f-469d-9eaa-1471ac1539e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:29:58.082519 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aaa45c-ac6f-469d-9eaa-1471ac1539e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:29:58.084133 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-60aaa45c-ac6f-469d-9eaa-1471ac1539e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:29:58.180944 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aaa45c-ac6f-469d-9eaa-1471ac1539e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:29:58.182303 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60aaa45c-ac6f-469d-9eaa-1471ac1539e1 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:29:58.186423 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:29:58.201858 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 590/1182] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:29:57 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:29:58.246167 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a75c3e-cb3d-49d4-a78b-9e7cb1210aa3 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:29:58.246456 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a75c3e-cb3d-49d4-a78b-9e7cb1210aa3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:29:58.246731 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a75c3e-cb3d-49d4-a78b-9e7cb1210aa3 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:29:58.248511 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 593/1183] 10.222.0.26 () {52 vars in 830 bytes} [Wed Nov 2 03:29:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 03:29:58.252135 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aaa45c-ac6f-469d-9eaa-1471ac1539e1 service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:29:58.253342 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-60aaa45c-ac6f-469d-9eaa-1471ac1539e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:29:58.350387 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60aaa45c-ac6f-469d-9eaa-1471ac1539e1 service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:29:58.443257 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ba6e3f-22a3-489c-8fbc-9776c734af69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:29:58.443897 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ba6e3f-22a3-489c-8fbc-9776c734af69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:29:58.444124 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ba6e3f-22a3-489c-8fbc-9776c734af69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:29:58.444394 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ba6e3f-22a3-489c-8fbc-9776c734af69 admin admin] PATH_INFO: `/v3/users/f693d67a771f48e3b459cde924d626cf` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:29:58.457752 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ba6e3f-22a3-489c-8fbc-9776c734af69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f693d67a771f48e3b459cde924d626cf)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:29:58.458769 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: WARNING py.warnings [None req-d3ba6e3f-22a3-489c-8fbc-9776c734af69 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 03:29:58.458769 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: warnings.warn(msg) Nov 02 03:29:58.458769 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: Nov 02 03:29:58.459989 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ba6e3f-22a3-489c-8fbc-9776c734af69 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:29:58.471257 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:29:58.525641 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d3ba6e3f-22a3-489c-8fbc-9776c734af69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f693d67a771f48e3b459cde924d626cf'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:29:58.526074 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d3ba6e3f-22a3-489c-8fbc-9776c734af69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f693d67a771f48e3b459cde924d626cf'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:29:58.526590 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d3ba6e3f-22a3-489c-8fbc-9776c734af69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f693d67a771f48e3b459cde924d626cf'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:29:58.549901 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d3ba6e3f-22a3-489c-8fbc-9776c734af69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f693d67a771f48e3b459cde924d626cf'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:29:58.559020 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d3ba6e3f-22a3-489c-8fbc-9776c734af69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f693d67a771f48e3b459cde924d626cf'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:29:58.579591 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 591/1184] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 03:29:58 2022] DELETE /identity/v3/users/f693d67a771f48e3b459cde924d626cf => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:29:58.584859 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f87d4-8b8e-40fa-902a-df7b892f0a72 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:29:58.585363 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f87d4-8b8e-40fa-902a-df7b892f0a72 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:29:58.585467 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f87d4-8b8e-40fa-902a-df7b892f0a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:29:58.641787 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-992f87d4-8b8e-40fa-902a-df7b892f0a72 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:29:58.642974 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-992f87d4-8b8e-40fa-902a-df7b892f0a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:29:58.718393 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:29:58.731060 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 594/1185] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:29:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:29:58.741264 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ba6e3f-22a3-489c-8fbc-9776c734af69 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:29:58.742743 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d3ba6e3f-22a3-489c-8fbc-9776c734af69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:29:58.841295 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ba6e3f-22a3-489c-8fbc-9776c734af69 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:29:58.895771 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:29:58.916914 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93991528-f4bd-4e3f-b42e-30b4b89b433b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:29:58.917528 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93991528-f4bd-4e3f-b42e-30b4b89b433b service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:29:58.917807 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93991528-f4bd-4e3f-b42e-30b4b89b433b service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:29:58.918086 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93991528-f4bd-4e3f-b42e-30b4b89b433b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:29:58.919675 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-93991528-f4bd-4e3f-b42e-30b4b89b433b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:29:59.027306 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93991528-f4bd-4e3f-b42e-30b4b89b433b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:29:59.028704 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93991528-f4bd-4e3f-b42e-30b4b89b433b service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:29:59.046981 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 592/1186] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:29:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:29:59.397226 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992f87d4-8b8e-40fa-902a-df7b892f0a72 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:29:59.398532 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-992f87d4-8b8e-40fa-902a-df7b892f0a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:29:59.410027 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992f87d4-8b8e-40fa-902a-df7b892f0a72 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:29:59.508076 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5edd6284-70a2-41bb-9f6a-d45ae8962ae6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:29:59.508711 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd6284-70a2-41bb-9f6a-d45ae8962ae6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:29:59.508991 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd6284-70a2-41bb-9f6a-d45ae8962ae6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:29:59.509262 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edd6284-70a2-41bb-9f6a-d45ae8962ae6 admin admin] PATH_INFO: `/v3/projects/9ceb9ab23fd84afc90fa0f39131238a0` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:29:59.519215 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edd6284-70a2-41bb-9f6a-d45ae8962ae6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9ceb9ab23fd84afc90fa0f39131238a0)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:29:59.520359 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: WARNING py.warnings [None req-5edd6284-70a2-41bb-9f6a-d45ae8962ae6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 03:29:59.520359 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: warnings.warn(msg) Nov 02 03:29:59.520359 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: Nov 02 03:29:59.521606 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5edd6284-70a2-41bb-9f6a-d45ae8962ae6 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:29:59.579316 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5edd6284-70a2-41bb-9f6a-d45ae8962ae6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9ceb9ab23fd84afc90fa0f39131238a0', 'request_id': 'req-5edd6284-70a2-41bb-9f6a-d45ae8962ae6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:29:59.592721 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5edd6284-70a2-41bb-9f6a-d45ae8962ae6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9ceb9ab23fd84afc90fa0f39131238a0', 'request_id': 'req-5edd6284-70a2-41bb-9f6a-d45ae8962ae6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:29:59.601826 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5edd6284-70a2-41bb-9f6a-d45ae8962ae6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9ceb9ab23fd84afc90fa0f39131238a0', 'request_id': 'req-5edd6284-70a2-41bb-9f6a-d45ae8962ae6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:29:59.625880 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5edd6284-70a2-41bb-9f6a-d45ae8962ae6 admin admin] The token cache is being invalidate because project 9ceb9ab23fd84afc90fa0f39131238a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 03:29:59.626168 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5edd6284-70a2-41bb-9f6a-d45ae8962ae6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:29:59.626593 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5edd6284-70a2-41bb-9f6a-d45ae8962ae6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:29:59.627970 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 595/1187] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 03:29:59 2022] DELETE /identity/v3/projects/9ceb9ab23fd84afc90fa0f39131238a0 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:30:09.044553 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=62 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:30:09.623882 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:31:25.899541 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ff3b3-919b-438e-b4c7-0a37c7e45127 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:31:25.900194 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ff3b3-919b-438e-b4c7-0a37c7e45127 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:31:25.900194 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2ff3b3-919b-438e-b4c7-0a37c7e45127 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:31:25.901472 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 593/1188] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 03:31:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:31:25.906798 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c454676b-c73c-4b84-82cb-6a6e45397046 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:31:25.907235 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c454676b-c73c-4b84-82cb-6a6e45397046 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:31:25.907338 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c454676b-c73c-4b84-82cb-6a6e45397046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:31:25.987631 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-c454676b-c73c-4b84-82cb-6a6e45397046 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:31:25.988867 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c454676b-c73c-4b84-82cb-6a6e45397046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:31:26.090741 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 596/1189] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 03:31:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:31:26.243709 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ff3b3-919b-438e-b4c7-0a37c7e45127 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:31:26.246065 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-4b2ff3b3-919b-438e-b4c7-0a37c7e45127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:31:26.366173 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2ff3b3-919b-438e-b4c7-0a37c7e45127 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:31:26.371813 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dc8164-cf99-4145-8e6a-99615638eb98 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:31:26.372445 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dc8164-cf99-4145-8e6a-99615638eb98 service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:31:26.372724 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dc8164-cf99-4145-8e6a-99615638eb98 service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:31:26.372996 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dc8164-cf99-4145-8e6a-99615638eb98 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:31:26.383369 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dc8164-cf99-4145-8e6a-99615638eb98 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:31:26.384963 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dc8164-cf99-4145-8e6a-99615638eb98 service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:31:26.396654 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 594/1190] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:31:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:31:27.468805 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25efd50-71dc-4963-b96d-53dac183e896 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:31:27.469343 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25efd50-71dc-4963-b96d-53dac183e896 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:31:27.469343 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25efd50-71dc-4963-b96d-53dac183e896 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:31:27.470706 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 597/1191] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 03:31:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:31:27.475427 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129de5d4-dd06-4b2e-b2c6-e9f0b1ac54fb None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:31:27.475835 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129de5d4-dd06-4b2e-b2c6-e9f0b1ac54fb None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:31:27.476008 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129de5d4-dd06-4b2e-b2c6-e9f0b1ac54fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:31:27.495561 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-129de5d4-dd06-4b2e-b2c6-e9f0b1ac54fb None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:31:27.496786 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-129de5d4-dd06-4b2e-b2c6-e9f0b1ac54fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:31:27.505127 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 595/1192] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 03:31:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:31:27.658975 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25efd50-71dc-4963-b96d-53dac183e896 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:31:27.673857 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25efd50-71dc-4963-b96d-53dac183e896 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:31:27.679168 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d138784-cd7d-4298-8eb1-228852b15565 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:31:27.679766 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d138784-cd7d-4298-8eb1-228852b15565 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:31:27.680082 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d138784-cd7d-4298-8eb1-228852b15565 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:31:27.680352 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d138784-cd7d-4298-8eb1-228852b15565 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:31:27.688914 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d138784-cd7d-4298-8eb1-228852b15565 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:31:27.689300 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0fd4c8-b555-4db6-97c4-a41e8dfb7c2a None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:31:27.689658 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0fd4c8-b555-4db6-97c4-a41e8dfb7c2a None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:31:27.690026 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0fd4c8-b555-4db6-97c4-a41e8dfb7c2a None None] PATH_INFO: `/v3` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:31:27.690480 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d138784-cd7d-4298-8eb1-228852b15565 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:31:27.691515 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 596/1193] 10.222.0.18 () {54 vars in 925 bytes} [Wed Nov 2 03:31:27 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:31:27.697843 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb608cb4-63af-417f-8eb4-150079de8bc0 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:31:27.698122 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb608cb4-63af-417f-8eb4-150079de8bc0 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:31:27.698513 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb608cb4-63af-417f-8eb4-150079de8bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:31:27.699903 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 598/1194] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:31:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:31:27.764064 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-bb608cb4-63af-417f-8eb4-150079de8bc0 None None] MFA Rules not processed for user `9b4b9bd741574ddabadd4b7651ebc885`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:31:27.765542 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-bb608cb4-63af-417f-8eb4-150079de8bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:31:27.823849 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:31:27.843546 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 597/1195] 10.222.0.18 () {58 vars in 1015 bytes} [Wed Nov 2 03:31:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 03:31:36.628233 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d54d571-afc0-4e86-bf95-e8fdae481ba3 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:31:36.628678 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d54d571-afc0-4e86-bf95-e8fdae481ba3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:31:36.628774 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d54d571-afc0-4e86-bf95-e8fdae481ba3 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:31:36.630137 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 599/1196] 10.222.0.26 () {54 vars in 961 bytes} [Wed Nov 2 03:31:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:31:36.635057 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9535f9-bad9-41fd-8ebf-3b7af888f8d7 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:31:36.635484 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9535f9-bad9-41fd-8ebf-3b7af888f8d7 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:31:36.635592 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9535f9-bad9-41fd-8ebf-3b7af888f8d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:31:36.654805 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-8a9535f9-bad9-41fd-8ebf-3b7af888f8d7 None None] MFA Rules not processed for user `9b4b9bd741574ddabadd4b7651ebc885`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:31:36.655994 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-8a9535f9-bad9-41fd-8ebf-3b7af888f8d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:31:36.664381 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 598/1197] 10.222.0.26 () {58 vars in 1058 bytes} [Wed Nov 2 03:31:36 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:31:36.671765 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d54d571-afc0-4e86-bf95-e8fdae481ba3 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:31:36.685363 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d54d571-afc0-4e86-bf95-e8fdae481ba3 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:31:36.690929 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9d75fb-4618-40fc-bedf-cc6c9bc11dde service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4b9bd741574ddabadd4b7651ebc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:31:36.691512 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9d75fb-4618-40fc-bedf-cc6c9bc11dde service placement] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:31:36.691791 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9d75fb-4618-40fc-bedf-cc6c9bc11dde service placement] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:31:36.692105 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9d75fb-4618-40fc-bedf-cc6c9bc11dde service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:31:36.700098 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9d75fb-4618-40fc-bedf-cc6c9bc11dde service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:31:36.701641 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9d75fb-4618-40fc-bedf-cc6c9bc11dde service placement] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:31:36.710977 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 600/1198] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:31:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 03:31:39.166395 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7616aec-0f81-40d6-94a6-dd58f0283067 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:31:39.166982 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7616aec-0f81-40d6-94a6-dd58f0283067 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:31:39.166982 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7616aec-0f81-40d6-94a6-dd58f0283067 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:31:39.168325 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 599/1199] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 03:31:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:31:39.173084 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2652237-44f5-42b8-a7b7-ce4c0dfbd851 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:31:39.173498 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2652237-44f5-42b8-a7b7-ce4c0dfbd851 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:31:39.173675 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2652237-44f5-42b8-a7b7-ce4c0dfbd851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:31:39.193835 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-b2652237-44f5-42b8-a7b7-ce4c0dfbd851 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:31:39.195034 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b2652237-44f5-42b8-a7b7-ce4c0dfbd851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:31:39.204467 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 601/1200] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 03:31:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:31:39.356510 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7616aec-0f81-40d6-94a6-dd58f0283067 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:31:39.370949 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7616aec-0f81-40d6-94a6-dd58f0283067 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:31:39.376931 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7009097b-542a-448c-9f07-0ca4e60debb6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:31:39.377521 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7009097b-542a-448c-9f07-0ca4e60debb6 service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:31:39.377804 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7009097b-542a-448c-9f07-0ca4e60debb6 service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:31:39.378072 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7009097b-542a-448c-9f07-0ca4e60debb6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:31:39.385984 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7009097b-542a-448c-9f07-0ca4e60debb6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:31:39.387550 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7009097b-542a-448c-9f07-0ca4e60debb6 service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:31:39.397399 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 600/1201] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:31:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 03:31:40.442500 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565cc233-cf27-42fc-b641-65dd0a689340 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:31:40.443021 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565cc233-cf27-42fc-b641-65dd0a689340 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:31:40.443021 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565cc233-cf27-42fc-b641-65dd0a689340 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:31:40.444328 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 602/1202] 127.0.0.1 () {54 vars in 901 bytes} [Wed Nov 2 03:31:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:31:40.449004 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2782e2-48ed-4633-9f32-1942ad08f4e3 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:31:40.449439 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2782e2-48ed-4633-9f32-1942ad08f4e3 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:31:40.449572 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2782e2-48ed-4633-9f32-1942ad08f4e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:31:40.469146 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-4a2782e2-48ed-4633-9f32-1942ad08f4e3 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:31:40.470929 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-4a2782e2-48ed-4633-9f32-1942ad08f4e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:31:40.478601 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 601/1203] 127.0.0.1 () {58 vars in 998 bytes} [Wed Nov 2 03:31:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:31:40.629276 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565cc233-cf27-42fc-b641-65dd0a689340 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:31:40.644086 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565cc233-cf27-42fc-b641-65dd0a689340 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:31:40.649739 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b714b439-3a4b-4d94-94eb-27718844afbd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:31:40.650319 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b714b439-3a4b-4d94-94eb-27718844afbd service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:31:40.650598 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b714b439-3a4b-4d94-94eb-27718844afbd service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:31:40.650870 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b714b439-3a4b-4d94-94eb-27718844afbd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:31:40.658762 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b714b439-3a4b-4d94-94eb-27718844afbd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:31:40.660386 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b714b439-3a4b-4d94-94eb-27718844afbd service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:31:40.669572 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 603/1204] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:31:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:31:44.592504 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae67e8-1e28-4461-9d4f-fdaffccc0fbe None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:31:44.593009 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae67e8-1e28-4461-9d4f-fdaffccc0fbe None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:31:44.593102 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ae67e8-1e28-4461-9d4f-fdaffccc0fbe None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:31:44.594431 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 602/1205] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 03:31:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:31:44.599750 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a6c3d9-024d-464e-bf9a-7838839bbab7 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:31:44.600274 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a6c3d9-024d-464e-bf9a-7838839bbab7 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:31:44.600377 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a6c3d9-024d-464e-bf9a-7838839bbab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:31:44.619357 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a2a6c3d9-024d-464e-bf9a-7838839bbab7 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:31:44.620848 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a2a6c3d9-024d-464e-bf9a-7838839bbab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:31:44.629022 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 604/1206] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 03:31:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:31:44.916618 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae67e8-1e28-4461-9d4f-fdaffccc0fbe None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:31:44.918298 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d3ae67e8-1e28-4461-9d4f-fdaffccc0fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:31:44.995003 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ae67e8-1e28-4461-9d4f-fdaffccc0fbe None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:31:44.998846 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee1b1e0-1c89-46c4-8032-83c0ff221d5a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:31:44.999448 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1b1e0-1c89-46c4-8032-83c0ff221d5a service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:31:44.999728 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1b1e0-1c89-46c4-8032-83c0ff221d5a service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:31:45.000029 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee1b1e0-1c89-46c4-8032-83c0ff221d5a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:31:45.009532 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee1b1e0-1c89-46c4-8032-83c0ff221d5a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:31:45.010947 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee1b1e0-1c89-46c4-8032-83c0ff221d5a service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:31:45.020291 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 603/1207] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:31:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:31:47.170626 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b9547-3fcc-415a-8966-cd998d2909c8 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:31:47.171258 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b9547-3fcc-415a-8966-cd998d2909c8 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:31:47.171258 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88b9547-3fcc-415a-8966-cd998d2909c8 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:31:47.173056 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 605/1208] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 03:31:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:31:47.178174 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2c4235-20bf-4fef-9223-5d2f651d735a None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:31:47.178624 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2c4235-20bf-4fef-9223-5d2f651d735a None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:31:47.178765 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2c4235-20bf-4fef-9223-5d2f651d735a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:31:47.198194 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-3c2c4235-20bf-4fef-9223-5d2f651d735a None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:31:47.199390 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-3c2c4235-20bf-4fef-9223-5d2f651d735a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:31:47.207913 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 604/1209] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 03:31:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:31:47.495856 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88b9547-3fcc-415a-8966-cd998d2909c8 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:31:47.497729 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a88b9547-3fcc-415a-8966-cd998d2909c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:31:47.511291 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88b9547-3fcc-415a-8966-cd998d2909c8 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:31:47.514854 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3592aff6-0ac2-44c6-b880-1689f44de488 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:31:47.515445 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3592aff6-0ac2-44c6-b880-1689f44de488 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:31:47.515719 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3592aff6-0ac2-44c6-b880-1689f44de488 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:31:47.516022 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3592aff6-0ac2-44c6-b880-1689f44de488 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:31:47.525604 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3592aff6-0ac2-44c6-b880-1689f44de488 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:31:47.527008 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3592aff6-0ac2-44c6-b880-1689f44de488 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:31:47.536533 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 606/1210] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:31:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:31:51.112069 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83810182-d52f-4c10-9d0f-b22a78e1b424 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:31:51.112628 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83810182-d52f-4c10-9d0f-b22a78e1b424 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:31:51.112628 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83810182-d52f-4c10-9d0f-b22a78e1b424 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:31:51.113956 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 605/1211] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 03:31:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:31:51.119090 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e03b7e-123e-4366-890b-78fb00a033e6 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:31:51.119537 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e03b7e-123e-4366-890b-78fb00a033e6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:31:51.119660 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e03b7e-123e-4366-890b-78fb00a033e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:31:51.128125 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:31:51.140734 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-b5e03b7e-123e-4366-890b-78fb00a033e6 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:31:51.142551 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b5e03b7e-123e-4366-890b-78fb00a033e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:31:51.150385 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 607/1212] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 03:31:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 03:31:51.455742 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83810182-d52f-4c10-9d0f-b22a78e1b424 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:31:51.457498 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-83810182-d52f-4c10-9d0f-b22a78e1b424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:31:51.534864 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83810182-d52f-4c10-9d0f-b22a78e1b424 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:31:51.538409 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85da9202-e620-42d6-99c0-3a039a7697c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:31:51.539010 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da9202-e620-42d6-99c0-3a039a7697c0 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:31:51.539284 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da9202-e620-42d6-99c0-3a039a7697c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:31:51.539553 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85da9202-e620-42d6-99c0-3a039a7697c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:31:51.549231 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85da9202-e620-42d6-99c0-3a039a7697c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:31:51.550665 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85da9202-e620-42d6-99c0-3a039a7697c0 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:31:51.560121 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 606/1213] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:31:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:31:53.695470 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e52c318-866f-4bfb-bd50-f5c024ecb006 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:31:53.696044 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e52c318-866f-4bfb-bd50-f5c024ecb006 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:31:53.696044 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e52c318-866f-4bfb-bd50-f5c024ecb006 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:31:53.697403 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 608/1214] 127.0.0.1 () {54 vars in 917 bytes} [Wed Nov 2 03:31:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:31:53.703036 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc46417-7758-414a-9b19-c57abb991c18 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:31:53.703454 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc46417-7758-414a-9b19-c57abb991c18 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:31:53.703613 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc46417-7758-414a-9b19-c57abb991c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:31:53.723333 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-9bc46417-7758-414a-9b19-c57abb991c18 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:31:53.724550 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-9bc46417-7758-414a-9b19-c57abb991c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:31:53.732250 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 607/1215] 127.0.0.1 () {58 vars in 1014 bytes} [Wed Nov 2 03:31:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:31:54.028659 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e52c318-866f-4bfb-bd50-f5c024ecb006 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:31:54.030320 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1e52c318-866f-4bfb-bd50-f5c024ecb006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:31:54.043144 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e52c318-866f-4bfb-bd50-f5c024ecb006 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:31:54.046607 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0879251-a037-4f61-971e-49bc59b2fcac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:31:54.047196 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0879251-a037-4f61-971e-49bc59b2fcac service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:31:54.047999 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0879251-a037-4f61-971e-49bc59b2fcac service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:31:54.048276 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0879251-a037-4f61-971e-49bc59b2fcac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:31:54.056999 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0879251-a037-4f61-971e-49bc59b2fcac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:31:54.058401 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0879251-a037-4f61-971e-49bc59b2fcac service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:31:54.068200 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 609/1216] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:31:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:32:03.718427 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=92 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:32:04.063537 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=8 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:17.875172 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b27dc4-b7b7-4b23-9f95-afcea645cf19 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:17.875768 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b27dc4-b7b7-4b23-9f95-afcea645cf19 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:17.875768 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b27dc4-b7b7-4b23-9f95-afcea645cf19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:17.899784 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-b6b27dc4-b7b7-4b23-9f95-afcea645cf19 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:17.901039 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-b6b27dc4-b7b7-4b23-9f95-afcea645cf19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:17.913193 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 608/1217] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:17.923126 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f37ee6-31f3-46e8-89f6-3fe2856ab924 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:17.923701 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f37ee6-31f3-46e8-89f6-3fe2856ab924 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:17.923701 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f37ee6-31f3-46e8-89f6-3fe2856ab924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:17.933007 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b1289d-217d-4fed-9c35-ee5a704d65a0 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:17.933334 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b1289d-217d-4fed-9c35-ee5a704d65a0 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:17.933571 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b1289d-217d-4fed-9c35-ee5a704d65a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:17.958594 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-e4f37ee6-31f3-46e8-89f6-3fe2856ab924 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:17.959803 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e4f37ee6-31f3-46e8-89f6-3fe2856ab924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:17.962246 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-46b1289d-217d-4fed-9c35-ee5a704d65a0 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:17.963436 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-46b1289d-217d-4fed-9c35-ee5a704d65a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:17.977391 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 609/1218] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Nov 02 03:33:17.978335 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 610/1219] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:17.979235 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89766d2-ceac-4dde-8ce3-d6be67fc6fa0 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:17.979545 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89766d2-ceac-4dde-8ce3-d6be67fc6fa0 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:17.979809 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89766d2-ceac-4dde-8ce3-d6be67fc6fa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:17.982751 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f37ee6-31f3-46e8-89f6-3fe2856ab924 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:17.997586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f37ee6-31f3-46e8-89f6-3fe2856ab924 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:18.002605 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02521691-7578-47a1-b4d4-fe664e462fa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:18.003228 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02521691-7578-47a1-b4d4-fe664e462fa0 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:18.003508 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02521691-7578-47a1-b4d4-fe664e462fa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:18.003779 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02521691-7578-47a1-b4d4-fe664e462fa0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:18.004726 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02521691-7578-47a1-b4d4-fe664e462fa0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:33:18.005027 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02521691-7578-47a1-b4d4-fe664e462fa0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:18.006816 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02521691-7578-47a1-b4d4-fe664e462fa0 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:18.015499 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-c89766d2-ceac-4dde-8ce3-d6be67fc6fa0 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:18.016757 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-c89766d2-ceac-4dde-8ce3-d6be67fc6fa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:18.029883 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 611/1220] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 03:33:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:33:18.031158 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d4c569-b599-4556-9915-549777b99673 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:18.031445 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d4c569-b599-4556-9915-549777b99673 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:18.031722 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d4c569-b599-4556-9915-549777b99673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:18.037413 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 610/1221] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:18.038674 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a43486-7ede-4885-af12-d1e39f828b95 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:18.038962 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a43486-7ede-4885-af12-d1e39f828b95 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:18.039241 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a43486-7ede-4885-af12-d1e39f828b95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:18.071823 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-87d4c569-b599-4556-9915-549777b99673 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:18.073085 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-87d4c569-b599-4556-9915-549777b99673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:18.081457 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 612/1222] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:18.082112 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-b6a43486-7ede-4885-af12-d1e39f828b95 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:18.082254 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d4c569-b599-4556-9915-549777b99673 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:18.083696 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-b6a43486-7ede-4885-af12-d1e39f828b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:18.092405 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d4c569-b599-4556-9915-549777b99673 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:18.097903 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d14ed6-4bb8-4753-b8cc-3bf3e2187293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:18.098521 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d14ed6-4bb8-4753-b8cc-3bf3e2187293 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:18.098895 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d14ed6-4bb8-4753-b8cc-3bf3e2187293 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:18.099174 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d14ed6-4bb8-4753-b8cc-3bf3e2187293 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:18.099810 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d14ed6-4bb8-4753-b8cc-3bf3e2187293 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:18.100974 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: WARNING py.warnings [None req-b7d14ed6-4bb8-4753-b8cc-3bf3e2187293 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 03:33:18.100974 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: warnings.warn(msg) Nov 02 03:33:18.100974 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: Nov 02 03:33:18.102199 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d14ed6-4bb8-4753-b8cc-3bf3e2187293 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:18.107291 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 611/1223] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:18.108897 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75719dd1-9ab4-411a-88c5-4b74a1f83803 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:18.109356 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75719dd1-9ab4-411a-88c5-4b74a1f83803 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:18.109784 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75719dd1-9ab4-411a-88c5-4b74a1f83803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:18.132407 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-75719dd1-9ab4-411a-88c5-4b74a1f83803 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:18.134717 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-75719dd1-9ab4-411a-88c5-4b74a1f83803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:18.144273 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 612/1224] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:18.144793 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 613/1225] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:33:18 2022] POST /identity/v3/projects => generated 385 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:33:18.145625 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d793a05a-cfc2-4252-b2c1-0fd991864574 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:18.145965 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d793a05a-cfc2-4252-b2c1-0fd991864574 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:18.146206 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d14ed6-4bb8-4753-b8cc-3bf3e2187293 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:18.146342 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d793a05a-cfc2-4252-b2c1-0fd991864574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:18.171488 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-d793a05a-cfc2-4252-b2c1-0fd991864574 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:18.172761 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d793a05a-cfc2-4252-b2c1-0fd991864574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:18.218478 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d14ed6-4bb8-4753-b8cc-3bf3e2187293 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:18.223304 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a87e6fb-b657-4dad-b2a2-9103de0c7f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:18.223909 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87e6fb-b657-4dad-b2a2-9103de0c7f2a admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:18.224218 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87e6fb-b657-4dad-b2a2-9103de0c7f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:18.224607 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a87e6fb-b657-4dad-b2a2-9103de0c7f2a admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:18.225241 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a87e6fb-b657-4dad-b2a2-9103de0c7f2a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:33:18.225653 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a87e6fb-b657-4dad-b2a2-9103de0c7f2a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:18.227266 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a87e6fb-b657-4dad-b2a2-9103de0c7f2a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:18.234712 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 613/1226] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:18.236155 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace045e9-93a7-43cb-a106-e300a9bf4771 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:18.236437 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace045e9-93a7-43cb-a106-e300a9bf4771 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:18.236708 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace045e9-93a7-43cb-a106-e300a9bf4771 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:18.248817 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 614/1227] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 03:33:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:33:18.248817 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba22d4-4d00-450f-a811-ac5d17621bf3 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:18.248817 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba22d4-4d00-450f-a811-ac5d17621bf3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:18.248817 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbba22d4-4d00-450f-a811-ac5d17621bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:18.262157 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-ace045e9-93a7-43cb-a106-e300a9bf4771 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:18.263356 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ace045e9-93a7-43cb-a106-e300a9bf4771 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:18.263812 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-cbba22d4-4d00-450f-a811-ac5d17621bf3 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:18.264995 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-cbba22d4-4d00-450f-a811-ac5d17621bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:18.271431 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 614/1228] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:18.280227 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc5cb5-dfd1-49b1-a91d-fe87b5a72b21 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:18.280227 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc5cb5-dfd1-49b1-a91d-fe87b5a72b21 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:18.280227 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10dc5cb5-dfd1-49b1-a91d-fe87b5a72b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:18.280536 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 615/1229] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:18.280536 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ea851-7b46-47f5-9e79-32f8d04fc744 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:18.280536 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ea851-7b46-47f5-9e79-32f8d04fc744 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:18.280536 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5ea851-7b46-47f5-9e79-32f8d04fc744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:18.297368 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-db5ea851-7b46-47f5-9e79-32f8d04fc744 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:18.298530 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-10dc5cb5-dfd1-49b1-a91d-fe87b5a72b21 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:18.298679 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-db5ea851-7b46-47f5-9e79-32f8d04fc744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:18.299683 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-10dc5cb5-dfd1-49b1-a91d-fe87b5a72b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:18.306725 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 616/1230] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:18.307484 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5ea851-7b46-47f5-9e79-32f8d04fc744 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:18.307681 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 615/1231] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:18.313243 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2064d327-f111-4e81-a2c4-c3eb3923d9f0 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:18.313529 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2064d327-f111-4e81-a2c4-c3eb3923d9f0 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:18.314368 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2064d327-f111-4e81-a2c4-c3eb3923d9f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:18.322160 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5ea851-7b46-47f5-9e79-32f8d04fc744 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:18.326792 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d001440f-d792-4496-9cc6-a13c0320c51f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:18.327428 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d001440f-d792-4496-9cc6-a13c0320c51f admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:18.327712 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d001440f-d792-4496-9cc6-a13c0320c51f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:18.328015 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d001440f-d792-4496-9cc6-a13c0320c51f admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:18.328676 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d001440f-d792-4496-9cc6-a13c0320c51f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:18.330633 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d001440f-d792-4496-9cc6-a13c0320c51f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:18.333273 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-2064d327-f111-4e81-a2c4-c3eb3923d9f0 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:18.334445 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-2064d327-f111-4e81-a2c4-c3eb3923d9f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:18.342149 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 616/1232] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:18.342909 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2064d327-f111-4e81-a2c4-c3eb3923d9f0 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:18.352998 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2064d327-f111-4e81-a2c4-c3eb3923d9f0 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:18.354146 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 617/1233] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:33:18 2022] POST /identity/v3/users => generated 293 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:33:18.354868 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d001440f-d792-4496-9cc6-a13c0320c51f admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:18.358667 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db6af68-a1cb-4c2e-b8b3-56b9ebebcb87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:18.359310 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6af68-a1cb-4c2e-b8b3-56b9ebebcb87 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:18.359804 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6af68-a1cb-4c2e-b8b3-56b9ebebcb87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:18.360114 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db6af68-a1cb-4c2e-b8b3-56b9ebebcb87 admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:18.361354 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db6af68-a1cb-4c2e-b8b3-56b9ebebcb87 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:33:18.362268 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db6af68-a1cb-4c2e-b8b3-56b9ebebcb87 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:18.363306 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: WARNING py.warnings [None req-7db6af68-a1cb-4c2e-b8b3-56b9ebebcb87 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Nov 02 03:33:18.363306 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: warnings.warn(msg) Nov 02 03:33:18.363306 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: Nov 02 03:33:18.364239 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db6af68-a1cb-4c2e-b8b3-56b9ebebcb87 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:18.365055 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d001440f-d792-4496-9cc6-a13c0320c51f admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:18.369764 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e961cd-dd0c-4561-8855-93c6667a8435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:18.370367 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e961cd-dd0c-4561-8855-93c6667a8435 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:18.370641 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e961cd-dd0c-4561-8855-93c6667a8435 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:18.370905 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e961cd-dd0c-4561-8855-93c6667a8435 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:18.371516 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e961cd-dd0c-4561-8855-93c6667a8435 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:18.373678 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e961cd-dd0c-4561-8855-93c6667a8435 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:18.376768 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 617/1234] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 03:33:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:33:18.377611 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db6af68-a1cb-4c2e-b8b3-56b9ebebcb87 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:18.388096 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db6af68-a1cb-4c2e-b8b3-56b9ebebcb87 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:18.392772 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61730d7f-2b98-4b95-937e-138ac5b84f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:18.393361 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61730d7f-2b98-4b95-937e-138ac5b84f9b admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:18.393671 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61730d7f-2b98-4b95-937e-138ac5b84f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:18.393894 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61730d7f-2b98-4b95-937e-138ac5b84f9b admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:18.394535 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61730d7f-2b98-4b95-937e-138ac5b84f9b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:33:18.394834 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61730d7f-2b98-4b95-937e-138ac5b84f9b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:18.396440 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61730d7f-2b98-4b95-937e-138ac5b84f9b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:18.401374 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 618/1235] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:33:18 2022] POST /identity/v3/projects => generated 403 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:33:18.402700 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c90bda-344d-4eae-9340-b7d23f953cc3 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:18.402993 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c90bda-344d-4eae-9340-b7d23f953cc3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:18.403268 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c90bda-344d-4eae-9340-b7d23f953cc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:18.409970 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 618/1236] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 03:33:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:33:18.410657 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61730d7f-2b98-4b95-937e-138ac5b84f9b admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:18.420472 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61730d7f-2b98-4b95-937e-138ac5b84f9b admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:18.430087 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-76c90bda-344d-4eae-9340-b7d23f953cc3 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:18.431636 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-76c90bda-344d-4eae-9340-b7d23f953cc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:18.473493 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:18.477276 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a24fd9-c044-4d2b-a228-1f843738bd64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:18.477896 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a24fd9-c044-4d2b-a228-1f843738bd64 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:18.478176 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a24fd9-c044-4d2b-a228-1f843738bd64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:18.478451 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a24fd9-c044-4d2b-a228-1f843738bd64 admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:18.479298 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a24fd9-c044-4d2b-a228-1f843738bd64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:33:18.479642 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a24fd9-c044-4d2b-a228-1f843738bd64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:18.481465 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a24fd9-c044-4d2b-a228-1f843738bd64 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:18.493409 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 619/1237] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 03:33:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:33:18.494757 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 619/1238] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:18.494895 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f627d43b-60ed-485c-9475-08b1c99ab44c None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:18.495005 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f627d43b-60ed-485c-9475-08b1c99ab44c None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:18.495341 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f627d43b-60ed-485c-9475-08b1c99ab44c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:18.495965 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f1424-ca12-45b9-93c4-4d13b1c95f7c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:18.496396 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f1424-ca12-45b9-93c4-4d13b1c95f7c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:18.496682 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8f1424-ca12-45b9-93c4-4d13b1c95f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:18.515362 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-f627d43b-60ed-485c-9475-08b1c99ab44c None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:18.516379 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-8c8f1424-ca12-45b9-93c4-4d13b1c95f7c None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:18.516596 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f627d43b-60ed-485c-9475-08b1c99ab44c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:18.518054 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8c8f1424-ca12-45b9-93c4-4d13b1c95f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:18.525430 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 620/1239] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:18.529615 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 620/1240] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:18.529615 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f1424-ca12-45b9-93c4-4d13b1c95f7c None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:18.530426 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f627d43b-60ed-485c-9475-08b1c99ab44c None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:18.539934 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8f1424-ca12-45b9-93c4-4d13b1c95f7c None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:18.541058 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f627d43b-60ed-485c-9475-08b1c99ab44c None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:18.544695 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8446801-04be-42f8-85f5-c99daba2b6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:18.545300 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8446801-04be-42f8-85f5-c99daba2b6ab admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:18.545582 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8446801-04be-42f8-85f5-c99daba2b6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:18.545709 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5b9ae8-e5f7-436c-a02e-928bcbbe136b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:18.545853 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8446801-04be-42f8-85f5-c99daba2b6ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:18.546224 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b9ae8-e5f7-436c-a02e-928bcbbe136b admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:18.546506 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b9ae8-e5f7-436c-a02e-928bcbbe136b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:18.546644 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8446801-04be-42f8-85f5-c99daba2b6ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:18.546833 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5b9ae8-e5f7-436c-a02e-928bcbbe136b admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:18.547414 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5b9ae8-e5f7-436c-a02e-928bcbbe136b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:18.548982 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8446801-04be-42f8-85f5-c99daba2b6ab admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:18.549581 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5b9ae8-e5f7-436c-a02e-928bcbbe136b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:18.551754 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:18.560751 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 621/1241] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:33:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:33:18.561530 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5b9ae8-e5f7-436c-a02e-928bcbbe136b admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:18.570943 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5b9ae8-e5f7-436c-a02e-928bcbbe136b admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:18.575685 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ed17b-7932-4bfe-bd45-fcbd56240ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:18.576297 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ed17b-7932-4bfe-bd45-fcbd56240ceb admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:18.576580 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ed17b-7932-4bfe-bd45-fcbd56240ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:18.576916 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0ed17b-7932-4bfe-bd45-fcbd56240ceb admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:18.577472 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0ed17b-7932-4bfe-bd45-fcbd56240ceb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:18.578467 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: WARNING py.warnings [None req-3b0ed17b-7932-4bfe-bd45-fcbd56240ceb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 03:33:18.578467 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: warnings.warn(msg) Nov 02 03:33:18.578467 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: Nov 02 03:33:18.582684 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 621/1242] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:33:18 2022] POST /identity/v3/projects => generated 389 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:33:18.582810 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0ed17b-7932-4bfe-bd45-fcbd56240ceb admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:18.582908 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8446801-04be-42f8-85f5-c99daba2b6ab admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:18.605830 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 622/1243] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:33:18 2022] POST /identity/v3/users => generated 302 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:33:18.605830 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ed17b-7932-4bfe-bd45-fcbd56240ceb admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:18.612978 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0ed17b-7932-4bfe-bd45-fcbd56240ceb admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:18.649519 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8446801-04be-42f8-85f5-c99daba2b6ab admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:18.654463 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da9c6cb-71f1-420c-ad66-580c385efc81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:18.655079 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da9c6cb-71f1-420c-ad66-580c385efc81 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:18.655362 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da9c6cb-71f1-420c-ad66-580c385efc81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:18.655638 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da9c6cb-71f1-420c-ad66-580c385efc81 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:18.656306 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da9c6cb-71f1-420c-ad66-580c385efc81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:18.658811 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da9c6cb-71f1-420c-ad66-580c385efc81 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:18.669071 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9109a6-c0e5-4d2d-8efc-9f1ae794e5a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:18.669676 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9109a6-c0e5-4d2d-8efc-9f1ae794e5a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:18.669953 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9109a6-c0e5-4d2d-8efc-9f1ae794e5a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:18.670224 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9109a6-c0e5-4d2d-8efc-9f1ae794e5a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:18.670854 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9109a6-c0e5-4d2d-8efc-9f1ae794e5a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:33:18.671150 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9109a6-c0e5-4d2d-8efc-9f1ae794e5a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:18.672747 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9109a6-c0e5-4d2d-8efc-9f1ae794e5a2 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:18.684587 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 623/1244] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 03:33:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:33:18.691657 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9109a6-c0e5-4d2d-8efc-9f1ae794e5a2 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:18.691786 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 622/1245] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:33:18 2022] POST /identity/v3/projects => generated 379 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:33:18.691786 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da9c6cb-71f1-420c-ad66-580c385efc81 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:18.694938 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9109a6-c0e5-4d2d-8efc-9f1ae794e5a2 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:18.750225 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9ec2f4-b50e-422b-895b-ce18b9740af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:18.750836 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ec2f4-b50e-422b-895b-ce18b9740af4 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:18.751111 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ec2f4-b50e-422b-895b-ce18b9740af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:18.751429 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ec2f4-b50e-422b-895b-ce18b9740af4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:18.752089 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9ec2f4-b50e-422b-895b-ce18b9740af4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:18.753950 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9ec2f4-b50e-422b-895b-ce18b9740af4 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:18.758148 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da9c6cb-71f1-420c-ad66-580c385efc81 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:18.762961 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90865a18-a1ac-4012-8a2d-39e7ea3942a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:18.763579 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90865a18-a1ac-4012-8a2d-39e7ea3942a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:18.763855 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90865a18-a1ac-4012-8a2d-39e7ea3942a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:18.764167 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90865a18-a1ac-4012-8a2d-39e7ea3942a6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:18.764812 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90865a18-a1ac-4012-8a2d-39e7ea3942a6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:33:18.765116 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90865a18-a1ac-4012-8a2d-39e7ea3942a6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:18.766727 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90865a18-a1ac-4012-8a2d-39e7ea3942a6 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:18.780415 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 623/1246] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 03:33:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:33:18.781425 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90865a18-a1ac-4012-8a2d-39e7ea3942a6 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:18.781640 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 624/1247] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:33:18 2022] POST /identity/v3/projects => generated 363 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:33:18.782461 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9ec2f4-b50e-422b-895b-ce18b9740af4 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:18.794280 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90865a18-a1ac-4012-8a2d-39e7ea3942a6 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:18.825896 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:18.846065 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9ec2f4-b50e-422b-895b-ce18b9740af4 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:18.850642 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b7c90f-2c69-43a5-9ec8-30a7f45e171a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:18.851114 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03917da6-0684-4868-b897-cd5dc2994158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:18.851241 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b7c90f-2c69-43a5-9ec8-30a7f45e171a admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:18.851508 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b7c90f-2c69-43a5-9ec8-30a7f45e171a admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:18.851775 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b7c90f-2c69-43a5-9ec8-30a7f45e171a admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:18.851872 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03917da6-0684-4868-b897-cd5dc2994158 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:18.852259 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03917da6-0684-4868-b897-cd5dc2994158 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:18.852434 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b7c90f-2c69-43a5-9ec8-30a7f45e171a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:18.852548 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03917da6-0684-4868-b897-cd5dc2994158 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:18.853173 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03917da6-0684-4868-b897-cd5dc2994158 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:33:18.853479 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03917da6-0684-4868-b897-cd5dc2994158 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:18.854019 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b7c90f-2c69-43a5-9ec8-30a7f45e171a admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:18.855148 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03917da6-0684-4868-b897-cd5dc2994158 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:18.855443 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:18.867986 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 625/1248] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:33:18 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:33:18.867986 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b7c90f-2c69-43a5-9ec8-30a7f45e171a admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:18.868427 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 624/1249] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 03:33:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:33:18.869317 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03917da6-0684-4868-b897-cd5dc2994158 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:18.875630 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b7c90f-2c69-43a5-9ec8-30a7f45e171a admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:18.881320 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03917da6-0684-4868-b897-cd5dc2994158 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:18.882955 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac6d030-f55e-4e55-a159-ba7fa73230e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:18.883558 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac6d030-f55e-4e55-a159-ba7fa73230e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:18.883817 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac6d030-f55e-4e55-a159-ba7fa73230e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:18.884122 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac6d030-f55e-4e55-a159-ba7fa73230e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:18.884743 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac6d030-f55e-4e55-a159-ba7fa73230e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:18.885854 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caae17d-ee4c-4400-a0dd-4114c27cc00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:18.886427 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caae17d-ee4c-4400-a0dd-4114c27cc00d admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:18.886621 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac6d030-f55e-4e55-a159-ba7fa73230e4 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:18.886728 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caae17d-ee4c-4400-a0dd-4114c27cc00d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:18.886961 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caae17d-ee4c-4400-a0dd-4114c27cc00d admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:18.887588 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caae17d-ee4c-4400-a0dd-4114c27cc00d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:18.889194 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caae17d-ee4c-4400-a0dd-4114c27cc00d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:18.901848 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 625/1250] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:33:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:33:18.902656 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caae17d-ee4c-4400-a0dd-4114c27cc00d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:18.908566 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 626/1251] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:33:18 2022] POST /identity/v3/users => generated 295 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:33:18.909214 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac6d030-f55e-4e55-a159-ba7fa73230e4 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:18.917277 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caae17d-ee4c-4400-a0dd-4114c27cc00d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:18.918532 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac6d030-f55e-4e55-a159-ba7fa73230e4 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:18.921875 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36823bf9-ae7e-4691-bf25-f56ba1b78d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:18.922458 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36823bf9-ae7e-4691-bf25-f56ba1b78d60 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:18.922733 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36823bf9-ae7e-4691-bf25-f56ba1b78d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:18.923000 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36823bf9-ae7e-4691-bf25-f56ba1b78d60 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:18.923642 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36823bf9-ae7e-4691-bf25-f56ba1b78d60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:18.924246 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5ceb4a-a1b7-4187-acd2-7f8b29e27198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:18.924720 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ceb4a-a1b7-4187-acd2-7f8b29e27198 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:18.925020 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ceb4a-a1b7-4187-acd2-7f8b29e27198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:18.925311 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ceb4a-a1b7-4187-acd2-7f8b29e27198 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:18.925574 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36823bf9-ae7e-4691-bf25-f56ba1b78d60 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:18.925916 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5ceb4a-a1b7-4187-acd2-7f8b29e27198 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:18.927901 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5ceb4a-a1b7-4187-acd2-7f8b29e27198 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:18.946702 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 626/1252] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:33:18 2022] POST /identity/v3/users => generated 290 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:33:18.947716 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36823bf9-ae7e-4691-bf25-f56ba1b78d60 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:18.959435 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 627/1253] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:33:18 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:33:18.959435 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5ceb4a-a1b7-4187-acd2-7f8b29e27198 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:18.959720 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36823bf9-ae7e-4691-bf25-f56ba1b78d60 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:19.017747 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a44d6cb-73c9-47aa-90a7-e58a5960e9d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:19.018362 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a44d6cb-73c9-47aa-90a7-e58a5960e9d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:19.018640 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a44d6cb-73c9-47aa-90a7-e58a5960e9d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:19.018910 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a44d6cb-73c9-47aa-90a7-e58a5960e9d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:19.019540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a44d6cb-73c9-47aa-90a7-e58a5960e9d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:19.020246 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5ceb4a-a1b7-4187-acd2-7f8b29e27198 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:19.021485 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a44d6cb-73c9-47aa-90a7-e58a5960e9d9 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:19.024964 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5ed95d-eaf2-41d2-9ec5-18f9355bb7e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:19.025594 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ed95d-eaf2-41d2-9ec5-18f9355bb7e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:19.025984 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ed95d-eaf2-41d2-9ec5-18f9355bb7e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:19.026260 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5ed95d-eaf2-41d2-9ec5-18f9355bb7e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:19.026882 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5ed95d-eaf2-41d2-9ec5-18f9355bb7e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:19.028818 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5ed95d-eaf2-41d2-9ec5-18f9355bb7e6 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:19.042655 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 627/1254] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:33:18 2022] POST /identity/v3/users => generated 282 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:33:19.043822 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a44d6cb-73c9-47aa-90a7-e58a5960e9d9 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:19.053787 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a44d6cb-73c9-47aa-90a7-e58a5960e9d9 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:19.059476 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6374e486-9aab-4820-8345-8f101a2450d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:19.060175 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6374e486-9aab-4820-8345-8f101a2450d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:19.060458 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6374e486-9aab-4820-8345-8f101a2450d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:19.060935 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6374e486-9aab-4820-8345-8f101a2450d9 admin admin] PATH_INFO: `/v3/projects/d7961886d30c449aad4197024105a951/users/a820200be2114b1db5d350ead6625bf1/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:19.064676 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 628/1255] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:33:18 2022] POST /identity/v3/projects => generated 393 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:33:19.064676 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5ed95d-eaf2-41d2-9ec5-18f9355bb7e6 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:19.077542 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6374e486-9aab-4820-8345-8f101a2450d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7961886d30c449aad4197024105a951, user_id=a820200be2114b1db5d350ead6625bf1, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:19.080818 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6374e486-9aab-4820-8345-8f101a2450d9 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:19.095890 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 628/1256] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:33:19 2022] PUT /identity/v3/projects/d7961886d30c449aad4197024105a951/users/a820200be2114b1db5d350ead6625bf1/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:19.096720 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6374e486-9aab-4820-8345-8f101a2450d9 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:19.126779 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5ed95d-eaf2-41d2-9ec5-18f9355bb7e6 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:19.159784 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6374e486-9aab-4820-8345-8f101a2450d9 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:19.164486 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc93bd31-2699-4eb4-aa45-419c0f7f1003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:19.165095 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93bd31-2699-4eb4-aa45-419c0f7f1003 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:19.165375 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93bd31-2699-4eb4-aa45-419c0f7f1003 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:19.165648 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc93bd31-2699-4eb4-aa45-419c0f7f1003 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:19.166284 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc93bd31-2699-4eb4-aa45-419c0f7f1003 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:19.167908 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc93bd31-2699-4eb4-aa45-419c0f7f1003 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:19.173822 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:19.179613 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 629/1257] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:33:19 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:33:19.180471 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc93bd31-2699-4eb4-aa45-419c0f7f1003 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:19.185588 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:19.189837 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91974eb3-9c88-4671-91b9-7072b84f6302 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:19.190456 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91974eb3-9c88-4671-91b9-7072b84f6302 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:19.190735 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91974eb3-9c88-4671-91b9-7072b84f6302 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:19.191008 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91974eb3-9c88-4671-91b9-7072b84f6302 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:19.191644 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91974eb3-9c88-4671-91b9-7072b84f6302 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:19.192477 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc93bd31-2699-4eb4-aa45-419c0f7f1003 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:19.193600 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91974eb3-9c88-4671-91b9-7072b84f6302 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:19.197622 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4c26d3-8465-4679-b0e1-63c79241290a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:19.198259 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c26d3-8465-4679-b0e1-63c79241290a admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:19.198542 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c26d3-8465-4679-b0e1-63c79241290a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:19.198815 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4c26d3-8465-4679-b0e1-63c79241290a admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:19.199457 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4c26d3-8465-4679-b0e1-63c79241290a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:19.201065 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4c26d3-8465-4679-b0e1-63c79241290a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:19.214393 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 630/1258] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:33:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:33:19.214393 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4c26d3-8465-4679-b0e1-63c79241290a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:19.221785 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4c26d3-8465-4679-b0e1-63c79241290a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:19.225580 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 629/1259] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:33:19 2022] POST /identity/v3/projects => generated 375 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:33:19.225580 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91974eb3-9c88-4671-91b9-7072b84f6302 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:19.281965 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7293841a-55a8-4ba4-8547-ba1bb4b40959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:19.282614 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7293841a-55a8-4ba4-8547-ba1bb4b40959 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:19.282895 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7293841a-55a8-4ba4-8547-ba1bb4b40959 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:19.283170 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7293841a-55a8-4ba4-8547-ba1bb4b40959 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:19.283808 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7293841a-55a8-4ba4-8547-ba1bb4b40959 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:19.284953 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91974eb3-9c88-4671-91b9-7072b84f6302 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:19.285428 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7293841a-55a8-4ba4-8547-ba1bb4b40959 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:19.289714 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f541dd-7d19-421d-8a60-c46f2bcf1dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:19.290316 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f541dd-7d19-421d-8a60-c46f2bcf1dce admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:19.290602 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f541dd-7d19-421d-8a60-c46f2bcf1dce admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:19.290875 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f541dd-7d19-421d-8a60-c46f2bcf1dce admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:19.291508 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f541dd-7d19-421d-8a60-c46f2bcf1dce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:19.293454 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f541dd-7d19-421d-8a60-c46f2bcf1dce admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:19.295346 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 631/1260] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:33:19 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:33:19.295980 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7293841a-55a8-4ba4-8547-ba1bb4b40959 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:19.306644 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7293841a-55a8-4ba4-8547-ba1bb4b40959 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:19.311434 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0271d489-2eb8-4774-bd94-fd5e67a0ad1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:19.312059 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271d489-2eb8-4774-bd94-fd5e67a0ad1f admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:19.312347 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271d489-2eb8-4774-bd94-fd5e67a0ad1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:19.312621 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0271d489-2eb8-4774-bd94-fd5e67a0ad1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:19.313253 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0271d489-2eb8-4774-bd94-fd5e67a0ad1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:19.315408 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0271d489-2eb8-4774-bd94-fd5e67a0ad1f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:19.318568 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 630/1261] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:33:19 2022] POST /identity/v3/users => generated 292 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:33:19.319395 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f541dd-7d19-421d-8a60-c46f2bcf1dce admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:19.327456 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 632/1262] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:33:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:33:19.328702 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bac7ac-44ea-46b5-b9e9-cb652979bf9a None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:19.328983 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bac7ac-44ea-46b5-b9e9-cb652979bf9a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:19.329259 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bac7ac-44ea-46b5-b9e9-cb652979bf9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:19.332215 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f541dd-7d19-421d-8a60-c46f2bcf1dce admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:19.337095 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4874c72a-1daa-4843-adc7-96e0b251f613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:19.337705 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4874c72a-1daa-4843-adc7-96e0b251f613 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:19.337984 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4874c72a-1daa-4843-adc7-96e0b251f613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:19.338260 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4874c72a-1daa-4843-adc7-96e0b251f613 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:19.338891 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4874c72a-1daa-4843-adc7-96e0b251f613 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:19.340882 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4874c72a-1daa-4843-adc7-96e0b251f613 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:19.356352 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-f7bac7ac-44ea-46b5-b9e9-cb652979bf9a None None] MFA Rules not processed for user `a820200be2114b1db5d350ead6625bf1`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:19.357553 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f7bac7ac-44ea-46b5-b9e9-cb652979bf9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:19.362765 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 631/1263] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:33:19 2022] POST /identity/v3/users => generated 297 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:33:19.363557 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4874c72a-1daa-4843-adc7-96e0b251f613 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:19.373287 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4874c72a-1daa-4843-adc7-96e0b251f613 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:19.378535 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7805a744-0bf6-4a88-a0a1-b50e2bd96b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:19.379279 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805a744-0bf6-4a88-a0a1-b50e2bd96b9f admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:19.379594 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805a744-0bf6-4a88-a0a1-b50e2bd96b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:19.379872 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7805a744-0bf6-4a88-a0a1-b50e2bd96b9f admin admin] PATH_INFO: `/v3/projects/8811221b05f8451f8448997951fdf35f/users/ce0536b28d3e457da55c4f4d1c618e4b/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:19.394631 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7805a744-0bf6-4a88-a0a1-b50e2bd96b9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8811221b05f8451f8448997951fdf35f, user_id=ce0536b28d3e457da55c4f4d1c618e4b, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:19.397332 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7805a744-0bf6-4a88-a0a1-b50e2bd96b9f admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:19.411261 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 632/1264] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:33:19 2022] PUT /identity/v3/projects/8811221b05f8451f8448997951fdf35f/users/ce0536b28d3e457da55c4f4d1c618e4b/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:19.412079 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7805a744-0bf6-4a88-a0a1-b50e2bd96b9f admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:19.430870 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 633/1265] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:19.431692 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bac7ac-44ea-46b5-b9e9-cb652979bf9a None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:19.442698 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bac7ac-44ea-46b5-b9e9-cb652979bf9a None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:19.483778 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7805a744-0bf6-4a88-a0a1-b50e2bd96b9f admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:19.489763 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406603e6-565e-4d39-b974-d1466a38158a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:19.490385 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406603e6-565e-4d39-b974-d1466a38158a admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:19.490662 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406603e6-565e-4d39-b974-d1466a38158a admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:19.490935 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406603e6-565e-4d39-b974-d1466a38158a admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:19.491575 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406603e6-565e-4d39-b974-d1466a38158a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:19.493202 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406603e6-565e-4d39-b974-d1466a38158a admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:19.503092 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ce736e-0512-4036-8e03-c3b8b314378a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:19.503715 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce736e-0512-4036-8e03-c3b8b314378a admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:19.503859 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 633/1266] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:33:19 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:33:19.504060 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce736e-0512-4036-8e03-c3b8b314378a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:19.504296 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ce736e-0512-4036-8e03-c3b8b314378a admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:19.504643 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406603e6-565e-4d39-b974-d1466a38158a admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:19.504923 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ce736e-0512-4036-8e03-c3b8b314378a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:19.506824 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ce736e-0512-4036-8e03-c3b8b314378a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:19.514122 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406603e6-565e-4d39-b974-d1466a38158a admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:19.517844 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:19.519230 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1a6e30-c678-4fd7-9cbe-80cd8a69db6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:19.519829 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a6e30-c678-4fd7-9cbe-80cd8a69db6f admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:19.520130 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a6e30-c678-4fd7-9cbe-80cd8a69db6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:19.520409 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a6e30-c678-4fd7-9cbe-80cd8a69db6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:19.521037 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1a6e30-c678-4fd7-9cbe-80cd8a69db6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:19.522625 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1a6e30-c678-4fd7-9cbe-80cd8a69db6f admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:19.529771 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 634/1267] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:33:19 2022] POST /identity/v3/users => generated 288 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:33:19.530810 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ce736e-0512-4036-8e03-c3b8b314378a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:19.536896 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 634/1268] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:33:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:33:19.537747 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1a6e30-c678-4fd7-9cbe-80cd8a69db6f admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:19.544467 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ce736e-0512-4036-8e03-c3b8b314378a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:19.547411 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1a6e30-c678-4fd7-9cbe-80cd8a69db6f admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:19.550395 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b947018-4750-4170-a290-459ae3ece342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:19.551005 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b947018-4750-4170-a290-459ae3ece342 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:19.551287 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b947018-4750-4170-a290-459ae3ece342 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:19.551560 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b947018-4750-4170-a290-459ae3ece342 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:19.552240 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b947018-4750-4170-a290-459ae3ece342 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:19.552645 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762105d6-9274-43d3-90b0-85d1242985d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:19.553344 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762105d6-9274-43d3-90b0-85d1242985d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:19.553687 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762105d6-9274-43d3-90b0-85d1242985d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:19.553857 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b947018-4750-4170-a290-459ae3ece342 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:19.554075 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762105d6-9274-43d3-90b0-85d1242985d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:19.554776 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762105d6-9274-43d3-90b0-85d1242985d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:19.556425 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762105d6-9274-43d3-90b0-85d1242985d6 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:19.557727 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:19.564279 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 635/1269] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:33:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:33:19.565106 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b947018-4750-4170-a290-459ae3ece342 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:19.571015 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 635/1270] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:33:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:33:19.572399 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5efb8be-28e9-42b5-bf6a-1335ccc0fe47 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:19.572679 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5efb8be-28e9-42b5-bf6a-1335ccc0fe47 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:19.572948 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5efb8be-28e9-42b5-bf6a-1335ccc0fe47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:19.577651 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b947018-4750-4170-a290-459ae3ece342 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:19.582344 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adedf789-f015-450e-9a1e-dfa31394aa24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:19.582947 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adedf789-f015-450e-9a1e-dfa31394aa24 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:19.583227 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adedf789-f015-450e-9a1e-dfa31394aa24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:19.583510 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adedf789-f015-450e-9a1e-dfa31394aa24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:19.584222 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adedf789-f015-450e-9a1e-dfa31394aa24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:19.585815 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adedf789-f015-450e-9a1e-dfa31394aa24 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:19.596955 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 636/1271] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:33:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:33:19.598157 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d386b04-5c3b-4d11-b570-76bf4ff1b092 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:19.598441 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d386b04-5c3b-4d11-b570-76bf4ff1b092 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:19.598714 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d386b04-5c3b-4d11-b570-76bf4ff1b092 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:19.600148 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 637/1272] 10.222.0.26 () {54 vars in 961 bytes} [Wed Nov 2 03:33:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:33:19.600686 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d386b04-5c3b-4d11-b570-76bf4ff1b092 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:19.603485 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-d5efb8be-28e9-42b5-bf6a-1335ccc0fe47 None None] MFA Rules not processed for user `ce0536b28d3e457da55c4f4d1c618e4b`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:19.604939 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d5efb8be-28e9-42b5-bf6a-1335ccc0fe47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:19.613275 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d386b04-5c3b-4d11-b570-76bf4ff1b092 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:19.617780 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d53b66-a764-46dd-831c-1e0898c5477b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:19.618407 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d53b66-a764-46dd-831c-1e0898c5477b admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:19.618688 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d53b66-a764-46dd-831c-1e0898c5477b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:19.618960 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d53b66-a764-46dd-831c-1e0898c5477b admin admin] PATH_INFO: `/v3/projects/d56f6d222a4d4411b8f4583c61f53129/users/f16c6a2cd8594037ad47475ed1b3161c/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:19.633638 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d53b66-a764-46dd-831c-1e0898c5477b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d56f6d222a4d4411b8f4583c61f53129, user_id=f16c6a2cd8594037ad47475ed1b3161c, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:19.636383 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d53b66-a764-46dd-831c-1e0898c5477b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:19.649468 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 638/1273] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:33:19 2022] PUT /identity/v3/projects/d56f6d222a4d4411b8f4583c61f53129/users/f16c6a2cd8594037ad47475ed1b3161c/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:19.650329 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d53b66-a764-46dd-831c-1e0898c5477b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:19.671303 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 636/1274] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:19 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:19.672194 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5efb8be-28e9-42b5-bf6a-1335ccc0fe47 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:19.683071 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5efb8be-28e9-42b5-bf6a-1335ccc0fe47 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:19.712327 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d53b66-a764-46dd-831c-1e0898c5477b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:19.717021 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15aea80-d7f5-49ef-9716-1268763b27d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:19.717654 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15aea80-d7f5-49ef-9716-1268763b27d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:19.717931 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15aea80-d7f5-49ef-9716-1268763b27d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:19.718201 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15aea80-d7f5-49ef-9716-1268763b27d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:19.718833 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15aea80-d7f5-49ef-9716-1268763b27d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:19.720984 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15aea80-d7f5-49ef-9716-1268763b27d5 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:19.730690 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 639/1275] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:33:19 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:33:19.731469 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15aea80-d7f5-49ef-9716-1268763b27d5 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:19.741567 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15aea80-d7f5-49ef-9716-1268763b27d5 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:19.743164 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520ed4ba-71d2-4962-bd0a-db07e138ca87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:19.743869 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ed4ba-71d2-4962-bd0a-db07e138ca87 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:19.744182 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ed4ba-71d2-4962-bd0a-db07e138ca87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:19.744460 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520ed4ba-71d2-4962-bd0a-db07e138ca87 admin admin] PATH_INFO: `/v3/projects/f7ed6d39b2e343548a094be05af24877/users/7fdb70174b404f4abf0bc220d5dd4bb1/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:19.746152 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b394c84d-ddf1-4b52-9f4d-833e15d48cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:19.746722 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b394c84d-ddf1-4b52-9f4d-833e15d48cb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:19.747259 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b394c84d-ddf1-4b52-9f4d-833e15d48cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:19.748262 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b394c84d-ddf1-4b52-9f4d-833e15d48cb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:19.748262 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b394c84d-ddf1-4b52-9f4d-833e15d48cb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:19.749723 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b394c84d-ddf1-4b52-9f4d-833e15d48cb1 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:19.759423 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 640/1276] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:33:19 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:33:19.760217 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b394c84d-ddf1-4b52-9f4d-833e15d48cb1 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:19.760451 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520ed4ba-71d2-4962-bd0a-db07e138ca87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7ed6d39b2e343548a094be05af24877, user_id=7fdb70174b404f4abf0bc220d5dd4bb1, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:19.763111 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520ed4ba-71d2-4962-bd0a-db07e138ca87 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:19.769748 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b394c84d-ddf1-4b52-9f4d-833e15d48cb1 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:19.776028 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 637/1277] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:33:19 2022] PUT /identity/v3/projects/f7ed6d39b2e343548a094be05af24877/users/7fdb70174b404f4abf0bc220d5dd4bb1/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:19.776818 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520ed4ba-71d2-4962-bd0a-db07e138ca87 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:19.831250 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed304ca-b0f5-4aa3-b10f-1da06d319c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:19.832188 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed304ca-b0f5-4aa3-b10f-1da06d319c20 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:19.832468 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed304ca-b0f5-4aa3-b10f-1da06d319c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:19.833271 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed304ca-b0f5-4aa3-b10f-1da06d319c20 admin admin] PATH_INFO: `/v3/projects/e834781d69d64f57a5da8e634428c0fb/users/cf5c58f066ca4419ae81aa9732eda236/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:19.836294 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520ed4ba-71d2-4962-bd0a-db07e138ca87 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:19.840863 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12c1867-2238-446d-a684-d6ac03f4bf55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:19.841457 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12c1867-2238-446d-a684-d6ac03f4bf55 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:19.841738 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12c1867-2238-446d-a684-d6ac03f4bf55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:19.841996 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12c1867-2238-446d-a684-d6ac03f4bf55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:19.842624 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12c1867-2238-446d-a684-d6ac03f4bf55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:19.844260 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12c1867-2238-446d-a684-d6ac03f4bf55 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:19.848170 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed304ca-b0f5-4aa3-b10f-1da06d319c20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e834781d69d64f57a5da8e634428c0fb, user_id=cf5c58f066ca4419ae81aa9732eda236, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:19.850819 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed304ca-b0f5-4aa3-b10f-1da06d319c20 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:19.854473 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 638/1278] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:33:19 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:33:19.855645 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22fedf-ddd8-4a25-8997-1bb5c39619b9 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:19.855926 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22fedf-ddd8-4a25-8997-1bb5c39619b9 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:19.856271 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da22fedf-ddd8-4a25-8997-1bb5c39619b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:19.864180 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:19.865746 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 641/1279] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:33:19 2022] PUT /identity/v3/projects/e834781d69d64f57a5da8e634428c0fb/users/cf5c58f066ca4419ae81aa9732eda236/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:19.867145 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6781b-6e5f-4478-810e-c213bfb1d1cc None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:19.867573 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6781b-6e5f-4478-810e-c213bfb1d1cc None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:19.867854 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6781b-6e5f-4478-810e-c213bfb1d1cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:19.887147 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-3ee6781b-6e5f-4478-810e-c213bfb1d1cc None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:19.888382 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3ee6781b-6e5f-4478-810e-c213bfb1d1cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:19.898090 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-da22fedf-ddd8-4a25-8997-1bb5c39619b9 None None] MFA Rules not processed for user `44961b06151c4da382fcd54dd1f1e146`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:19.899252 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-da22fedf-ddd8-4a25-8997-1bb5c39619b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:19.900691 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:19.950179 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 642/1280] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:19.951130 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee6781b-6e5f-4478-810e-c213bfb1d1cc None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:19.960212 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee6781b-6e5f-4478-810e-c213bfb1d1cc None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:19.964810 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b95f5f5-8960-45c1-a8c4-78973a38809f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:19.965418 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b95f5f5-8960-45c1-a8c4-78973a38809f admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:19.965702 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b95f5f5-8960-45c1-a8c4-78973a38809f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:19.965974 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b95f5f5-8960-45c1-a8c4-78973a38809f admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:19.966608 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b95f5f5-8960-45c1-a8c4-78973a38809f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:19.968565 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b95f5f5-8960-45c1-a8c4-78973a38809f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:19.970409 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 639/1281] 10.222.0.26 () {58 vars in 1058 bytes} [Wed Nov 2 03:33:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:19.971244 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da22fedf-ddd8-4a25-8997-1bb5c39619b9 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:19.981428 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da22fedf-ddd8-4a25-8997-1bb5c39619b9 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:19.986916 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608ee987-32c5-4466-b6b9-6a8ef840e189 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:19.987523 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ee987-32c5-4466-b6b9-6a8ef840e189 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:19.987867 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ee987-32c5-4466-b6b9-6a8ef840e189 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:19.988130 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608ee987-32c5-4466-b6b9-6a8ef840e189 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:19.988769 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608ee987-32c5-4466-b6b9-6a8ef840e189 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:19.990333 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608ee987-32c5-4466-b6b9-6a8ef840e189 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:19.996390 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 643/1282] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:33:19 2022] POST /identity/v3/projects => generated 405 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:33:19.997557 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b95f5f5-8960-45c1-a8c4-78973a38809f admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:20.001057 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 640/1283] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:33:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:33:20.002808 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e4fe2-96b1-4bbb-92ed-a8b2a9157f5c None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:20.003432 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e4fe2-96b1-4bbb-92ed-a8b2a9157f5c None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:20.003432 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e4fe2-96b1-4bbb-92ed-a8b2a9157f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:20.022172 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-129e4fe2-96b1-4bbb-92ed-a8b2a9157f5c None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:20.023374 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-129e4fe2-96b1-4bbb-92ed-a8b2a9157f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:20.058809 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b95f5f5-8960-45c1-a8c4-78973a38809f admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:20.063673 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f13ec5-25e4-41e4-8a92-581733a501af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:20.064885 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f13ec5-25e4-41e4-8a92-581733a501af admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:20.065166 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f13ec5-25e4-41e4-8a92-581733a501af admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:20.065445 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f13ec5-25e4-41e4-8a92-581733a501af admin admin] PATH_INFO: `/v3/projects/10a674b48d484716a8a79d518ea0d579/users/7984dc1d452a4c998b44e6d26a299df8/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:20.080150 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f13ec5-25e4-41e4-8a92-581733a501af admin admin] RBAC: Authorizing `identity:create_grant(project_id=10a674b48d484716a8a79d518ea0d579, user_id=7984dc1d452a4c998b44e6d26a299df8, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:20.082809 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f13ec5-25e4-41e4-8a92-581733a501af admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:20.084569 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 641/1284] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:20.085438 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129e4fe2-96b1-4bbb-92ed-a8b2a9157f5c None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:20.095768 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129e4fe2-96b1-4bbb-92ed-a8b2a9157f5c None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:20.098325 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 644/1285] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:33:19 2022] PUT /identity/v3/projects/10a674b48d484716a8a79d518ea0d579/users/7984dc1d452a4c998b44e6d26a299df8/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:20.099670 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba4f05-5773-483a-bbdf-99a943b321a8 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:20.099987 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba4f05-5773-483a-bbdf-99a943b321a8 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:20.100261 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ba4f05-5773-483a-bbdf-99a943b321a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:20.128021 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-81ba4f05-5773-483a-bbdf-99a943b321a8 None None] MFA Rules not processed for user `cf5c58f066ca4419ae81aa9732eda236`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:20.129221 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-81ba4f05-5773-483a-bbdf-99a943b321a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:20.155122 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cc9a8f-f7c8-4014-aa82-e17b50d6942e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:20.155790 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cc9a8f-f7c8-4014-aa82-e17b50d6942e admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:20.156220 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cc9a8f-f7c8-4014-aa82-e17b50d6942e admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:20.156557 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cc9a8f-f7c8-4014-aa82-e17b50d6942e admin admin] PATH_INFO: `/v3/projects/c3a4704c59a141a2a1abbb5abb65c994/users/eed64456ec5c42b08df447e9a537c4d0/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:20.170457 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cc9a8f-f7c8-4014-aa82-e17b50d6942e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3a4704c59a141a2a1abbb5abb65c994, user_id=eed64456ec5c42b08df447e9a537c4d0, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:20.173723 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cc9a8f-f7c8-4014-aa82-e17b50d6942e admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:20.176692 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:20.188181 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 642/1286] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:33:20 2022] PUT /identity/v3/projects/c3a4704c59a141a2a1abbb5abb65c994/users/eed64456ec5c42b08df447e9a537c4d0/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:20.189515 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cc9a8f-f7c8-4014-aa82-e17b50d6942e admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:20.197894 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 645/1287] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:20 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:20.198727 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ba4f05-5773-483a-bbdf-99a943b321a8 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:20.208287 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:20.245328 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cc9a8f-f7c8-4014-aa82-e17b50d6942e admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:20.249227 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9af27c5-3187-4719-b5b1-4447017c8a49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:20.249826 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af27c5-3187-4719-b5b1-4447017c8a49 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:20.250105 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af27c5-3187-4719-b5b1-4447017c8a49 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:20.250374 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af27c5-3187-4719-b5b1-4447017c8a49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:20.275143 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ba4f05-5773-483a-bbdf-99a943b321a8 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:20.310823 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9af27c5-3187-4719-b5b1-4447017c8a49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:20.312334 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9af27c5-3187-4719-b5b1-4447017c8a49 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:20.322436 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 643/1288] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:33:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:33:20.323295 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9af27c5-3187-4719-b5b1-4447017c8a49 service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:20.332475 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9af27c5-3187-4719-b5b1-4447017c8a49 service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:20.339197 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed394de3-083e-4827-9221-895efb0684b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:20.340133 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed394de3-083e-4827-9221-895efb0684b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:20.340133 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed394de3-083e-4827-9221-895efb0684b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:20.340425 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed394de3-083e-4827-9221-895efb0684b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:20.341446 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed394de3-083e-4827-9221-895efb0684b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:20.341738 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c720ad6f-92d4-4829-be26-e39c91246918 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:20.342339 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c720ad6f-92d4-4829-be26-e39c91246918 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:20.342716 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c720ad6f-92d4-4829-be26-e39c91246918 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:20.343228 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c720ad6f-92d4-4829-be26-e39c91246918 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:20.343707 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed394de3-083e-4827-9221-895efb0684b4 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:20.369550 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 644/1289] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:33:20 2022] POST /identity/v3/users => generated 309 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:33:20.370162 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed394de3-083e-4827-9221-895efb0684b4 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:20.382388 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed394de3-083e-4827-9221-895efb0684b4 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:20.387825 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e67587-c3d1-422d-9aa3-44c2a69db01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:20.388233 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e67587-c3d1-422d-9aa3-44c2a69db01c admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:20.388466 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e67587-c3d1-422d-9aa3-44c2a69db01c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:20.388885 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e67587-c3d1-422d-9aa3-44c2a69db01c admin admin] PATH_INFO: `/v3/projects/85403204e27c40ffb0155dd8aba3d77b/users/18355f4225c9427dbb38fde84115d7b2/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:20.404895 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e67587-c3d1-422d-9aa3-44c2a69db01c admin admin] RBAC: Authorizing `identity:create_grant(project_id=85403204e27c40ffb0155dd8aba3d77b, user_id=18355f4225c9427dbb38fde84115d7b2, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:20.408288 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e67587-c3d1-422d-9aa3-44c2a69db01c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:20.410759 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c720ad6f-92d4-4829-be26-e39c91246918 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:20.412994 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c720ad6f-92d4-4829-be26-e39c91246918 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:20.423836 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 645/1290] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:33:20 2022] PUT /identity/v3/projects/85403204e27c40ffb0155dd8aba3d77b/users/18355f4225c9427dbb38fde84115d7b2/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:20.425212 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e67587-c3d1-422d-9aa3-44c2a69db01c admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:20.430521 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 646/1291] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:33:20 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:33:20.441399 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e411adbe-9510-4284-9cec-bf2a11b87ea8 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:20.441674 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e411adbe-9510-4284-9cec-bf2a11b87ea8 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:20.441943 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e411adbe-9510-4284-9cec-bf2a11b87ea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:20.464450 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-e411adbe-9510-4284-9cec-bf2a11b87ea8 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:20.465843 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e411adbe-9510-4284-9cec-bf2a11b87ea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:20.497278 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e67587-c3d1-422d-9aa3-44c2a69db01c admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:20.501689 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96f79d7-c7cb-44ae-9937-ba67583e9962 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:20.502401 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f79d7-c7cb-44ae-9937-ba67583e9962 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:20.502740 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f79d7-c7cb-44ae-9937-ba67583e9962 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:20.503041 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f79d7-c7cb-44ae-9937-ba67583e9962 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:20.514182 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:20.525185 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:20.549733 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 647/1292] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:20.554330 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d213286-9faa-4865-803e-192ef177790f None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:20.554619 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d213286-9faa-4865-803e-192ef177790f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:20.554902 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d213286-9faa-4865-803e-192ef177790f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:20.580736 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-1d213286-9faa-4865-803e-192ef177790f None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:20.582153 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1d213286-9faa-4865-803e-192ef177790f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:20.591518 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 648/1293] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:20.596148 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d213286-9faa-4865-803e-192ef177790f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:20.613231 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d213286-9faa-4865-803e-192ef177790f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:20.617864 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96f79d7-c7cb-44ae-9937-ba67583e9962 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:20.619548 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96f79d7-c7cb-44ae-9937-ba67583e9962 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:20.629222 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eb5997-7f58-4291-ad08-0609d0806025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:20.630499 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb5997-7f58-4291-ad08-0609d0806025 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:20.630958 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb5997-7f58-4291-ad08-0609d0806025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:20.631381 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 646/1294] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:33:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:33:20.631667 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1eb5997-7f58-4291-ad08-0609d0806025 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:20.632852 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96f79d7-c7cb-44ae-9937-ba67583e9962 service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:20.633696 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eb5997-7f58-4291-ad08-0609d0806025 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:20.636128 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1eb5997-7f58-4291-ad08-0609d0806025 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:20.644082 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96f79d7-c7cb-44ae-9937-ba67583e9962 service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:20.649531 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a982da99-4b99-4348-966e-6e4481398527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:20.650534 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982da99-4b99-4348-966e-6e4481398527 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:20.651086 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982da99-4b99-4348-966e-6e4481398527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:20.651526 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982da99-4b99-4348-966e-6e4481398527 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:20.652355 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a982da99-4b99-4348-966e-6e4481398527 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:20.654217 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a982da99-4b99-4348-966e-6e4481398527 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:20.670478 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 647/1295] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:33:20 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:33:20.672190 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aacfbfa-a91b-4096-9752-6fab6f5e1616 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:20.672805 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aacfbfa-a91b-4096-9752-6fab6f5e1616 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:20.673899 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aacfbfa-a91b-4096-9752-6fab6f5e1616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:20.685615 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 649/1296] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:33:20 2022] POST /identity/v3/projects => generated 363 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:33:20.687443 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eb5997-7f58-4291-ad08-0609d0806025 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:20.703326 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-7aacfbfa-a91b-4096-9752-6fab6f5e1616 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:20.705170 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-7aacfbfa-a91b-4096-9752-6fab6f5e1616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:20.805755 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 648/1297] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:20.806710 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aacfbfa-a91b-4096-9752-6fab6f5e1616 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:20.807877 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1eb5997-7f58-4291-ad08-0609d0806025 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:20.819307 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aacfbfa-a91b-4096-9752-6fab6f5e1616 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:20.823762 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1227b767-b03f-4248-a1f2-34e4a39080b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:20.824372 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa21a4e-d4a5-441e-b776-0f8e0e682bf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:20.824607 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1227b767-b03f-4248-a1f2-34e4a39080b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:20.824983 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1227b767-b03f-4248-a1f2-34e4a39080b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:20.825374 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1227b767-b03f-4248-a1f2-34e4a39080b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:20.825851 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa21a4e-d4a5-441e-b776-0f8e0e682bf6 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:20.826371 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa21a4e-d4a5-441e-b776-0f8e0e682bf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:20.826807 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa21a4e-d4a5-441e-b776-0f8e0e682bf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:20.841120 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1227b767-b03f-4248-a1f2-34e4a39080b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:20.842343 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa21a4e-d4a5-441e-b776-0f8e0e682bf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:20.843343 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1227b767-b03f-4248-a1f2-34e4a39080b2 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:20.843986 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa21a4e-d4a5-441e-b776-0f8e0e682bf6 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:20.855144 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 649/1298] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:33:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:33:20.855657 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 650/1299] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:33:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:33:20.856134 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1227b767-b03f-4248-a1f2-34e4a39080b2 service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:20.856612 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa21a4e-d4a5-441e-b776-0f8e0e682bf6 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:20.866801 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1227b767-b03f-4248-a1f2-34e4a39080b2 service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:20.870051 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa21a4e-d4a5-441e-b776-0f8e0e682bf6 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:20.872392 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f02983-b48e-4e99-ae0b-b05d48053a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:20.873083 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f02983-b48e-4e99-ae0b-b05d48053a53 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:20.873430 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f02983-b48e-4e99-ae0b-b05d48053a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:20.873841 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f02983-b48e-4e99-ae0b-b05d48053a53 admin admin] PATH_INFO: `/v3/projects/0b8c8e9a2d0c4d9c93aa6636ecd72557/users/53acb1b88afe49a7a90075826f46d62e/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:20.875431 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2989c51-f31b-4030-94e0-9417b5acc7ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:20.876222 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2989c51-f31b-4030-94e0-9417b5acc7ec admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:20.876620 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2989c51-f31b-4030-94e0-9417b5acc7ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:20.876951 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2989c51-f31b-4030-94e0-9417b5acc7ec admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:20.877584 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2989c51-f31b-4030-94e0-9417b5acc7ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:20.879553 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2989c51-f31b-4030-94e0-9417b5acc7ec admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:20.890607 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f02983-b48e-4e99-ae0b-b05d48053a53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b8c8e9a2d0c4d9c93aa6636ecd72557, user_id=53acb1b88afe49a7a90075826f46d62e, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:20.894143 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f02983-b48e-4e99-ae0b-b05d48053a53 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:20.907462 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 651/1300] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:33:20 2022] POST /identity/v3/users => generated 288 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:33:20.908868 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a127c-8f62-4460-850f-8c1e2bf7fc35 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:20.909236 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a127c-8f62-4460-850f-8c1e2bf7fc35 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:20.909576 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a127c-8f62-4460-850f-8c1e2bf7fc35 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:20.911085 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 650/1301] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:33:20 2022] PUT /identity/v3/projects/0b8c8e9a2d0c4d9c93aa6636ecd72557/users/53acb1b88afe49a7a90075826f46d62e/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:20.912316 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f02983-b48e-4e99-ae0b-b05d48053a53 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:20.915585 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 652/1302] 10.222.0.26 () {54 vars in 912 bytes} [Wed Nov 2 03:33:20 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:33:20.916536 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196a127c-8f62-4460-850f-8c1e2bf7fc35 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:20.981705 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f02983-b48e-4e99-ae0b-b05d48053a53 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:20.986053 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21088663-fd86-47f6-b708-2b11d98b00ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:20.987306 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21088663-fd86-47f6-b708-2b11d98b00ba service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:20.987759 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21088663-fd86-47f6-b708-2b11d98b00ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:20.988249 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21088663-fd86-47f6-b708-2b11d98b00ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:20.992755 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196a127c-8f62-4460-850f-8c1e2bf7fc35 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:20.998418 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21088663-fd86-47f6-b708-2b11d98b00ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:20.999875 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21088663-fd86-47f6-b708-2b11d98b00ba service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:21.000033 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe841e76-cfcb-488c-8c7b-a092417ef4f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:21.000630 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe841e76-cfcb-488c-8c7b-a092417ef4f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:21.000976 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe841e76-cfcb-488c-8c7b-a092417ef4f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:21.001352 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe841e76-cfcb-488c-8c7b-a092417ef4f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:21.002182 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe841e76-cfcb-488c-8c7b-a092417ef4f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:21.003812 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe841e76-cfcb-488c-8c7b-a092417ef4f4 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:21.011381 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 651/1303] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:33:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:33:21.012062 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21088663-fd86-47f6-b708-2b11d98b00ba service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:21.015718 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 653/1304] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:33:20 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:33:21.020909 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3bed3f-b6bb-4279-b614-49abe56375d3 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:21.022475 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3bed3f-b6bb-4279-b614-49abe56375d3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:21.022475 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3bed3f-b6bb-4279-b614-49abe56375d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:21.026302 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21088663-fd86-47f6-b708-2b11d98b00ba service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:21.033748 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9ba4b8-5eff-4432-95d4-da9fa5c6e36b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:21.034391 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ba4b8-5eff-4432-95d4-da9fa5c6e36b admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:21.034669 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ba4b8-5eff-4432-95d4-da9fa5c6e36b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:21.034936 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9ba4b8-5eff-4432-95d4-da9fa5c6e36b admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:21.035581 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9ba4b8-5eff-4432-95d4-da9fa5c6e36b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:21.037206 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9ba4b8-5eff-4432-95d4-da9fa5c6e36b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:21.040650 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:21.048369 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-8f3bed3f-b6bb-4279-b614-49abe56375d3 None None] MFA Rules not processed for user `44961b06151c4da382fcd54dd1f1e146`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:21.050110 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8f3bed3f-b6bb-4279-b614-49abe56375d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:21.054523 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 652/1305] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:33:21 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:33:21.056356 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9ba4b8-5eff-4432-95d4-da9fa5c6e36b admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:21.070424 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9ba4b8-5eff-4432-95d4-da9fa5c6e36b admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:21.092489 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73748bf2-d036-4294-9aea-da9e5860d5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:21.093437 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73748bf2-d036-4294-9aea-da9e5860d5e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:21.093864 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73748bf2-d036-4294-9aea-da9e5860d5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:21.094284 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73748bf2-d036-4294-9aea-da9e5860d5e0 admin admin] PATH_INFO: `/v3/projects/34b8d99c4a214eeb99f7f839afffffd5/users/254d1ba2b4b140769de8785b9f3ddae6/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:21.107618 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:21.120906 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73748bf2-d036-4294-9aea-da9e5860d5e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34b8d99c4a214eeb99f7f839afffffd5, user_id=254d1ba2b4b140769de8785b9f3ddae6, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:21.126987 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73748bf2-d036-4294-9aea-da9e5860d5e0 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:21.148520 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 653/1306] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:33:21 2022] PUT /identity/v3/projects/34b8d99c4a214eeb99f7f839afffffd5/users/254d1ba2b4b140769de8785b9f3ddae6/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:21.152531 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73748bf2-d036-4294-9aea-da9e5860d5e0 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:21.164790 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 654/1307] 10.222.0.26 () {58 vars in 1009 bytes} [Wed Nov 2 03:33:21 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:21.166316 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3bed3f-b6bb-4279-b614-49abe56375d3 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:21.179849 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3bed3f-b6bb-4279-b614-49abe56375d3 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:21.237506 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73748bf2-d036-4294-9aea-da9e5860d5e0 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:21.245139 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f4e8bf-e84c-40de-84a6-3e08d1524c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:21.245859 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4e8bf-e84c-40de-84a6-3e08d1524c12 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:21.246162 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4e8bf-e84c-40de-84a6-3e08d1524c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:21.246497 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f4e8bf-e84c-40de-84a6-3e08d1524c12 admin admin] PATH_INFO: `/v3/projects/0b8c8e9a2d0c4d9c93aa6636ecd72557/users/53acb1b88afe49a7a90075826f46d62e/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:21.263547 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f4e8bf-e84c-40de-84a6-3e08d1524c12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b8c8e9a2d0c4d9c93aa6636ecd72557, user_id=53acb1b88afe49a7a90075826f46d62e, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:21.271594 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f4e8bf-e84c-40de-84a6-3e08d1524c12 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:21.285751 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9391cd-c4ce-45fb-9f9d-bf1736eac769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:21.286552 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9391cd-c4ce-45fb-9f9d-bf1736eac769 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:21.286839 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9391cd-c4ce-45fb-9f9d-bf1736eac769 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:21.287116 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9391cd-c4ce-45fb-9f9d-bf1736eac769 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:21.287864 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9391cd-c4ce-45fb-9f9d-bf1736eac769 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:21.291683 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9391cd-c4ce-45fb-9f9d-bf1736eac769 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:21.311560 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 655/1309] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:33:21 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:33:21.311560 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9391cd-c4ce-45fb-9f9d-bf1736eac769 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:21.312500 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 654/1309] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:33:21 2022] PUT /identity/v3/projects/0b8c8e9a2d0c4d9c93aa6636ecd72557/users/53acb1b88afe49a7a90075826f46d62e/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:21.316119 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f4e8bf-e84c-40de-84a6-3e08d1524c12 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:21.386636 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9391cd-c4ce-45fb-9f9d-bf1736eac769 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:21.440696 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f4e8bf-e84c-40de-84a6-3e08d1524c12 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:21.446330 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e244c14-3e9e-4025-b499-5bf0a2ab4d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:21.447285 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e244c14-3e9e-4025-b499-5bf0a2ab4d4e admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:21.447682 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e244c14-3e9e-4025-b499-5bf0a2ab4d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:21.448442 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e244c14-3e9e-4025-b499-5bf0a2ab4d4e admin admin] PATH_INFO: `/v3/projects/34b8d99c4a214eeb99f7f839afffffd5/users/254d1ba2b4b140769de8785b9f3ddae6/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:21.449798 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50fff7a-dbd8-4e37-b032-5af3603b7a52 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:21.450551 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50fff7a-dbd8-4e37-b032-5af3603b7a52 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:21.450905 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50fff7a-dbd8-4e37-b032-5af3603b7a52 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:21.451211 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50fff7a-dbd8-4e37-b032-5af3603b7a52 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:21.451492 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e244c14-3e9e-4025-b499-5bf0a2ab4d4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=34b8d99c4a214eeb99f7f839afffffd5, user_id=254d1ba2b4b140769de8785b9f3ddae6, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:21.455674 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e244c14-3e9e-4025-b499-5bf0a2ab4d4e admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:21.475323 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 655/1310] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:33:21 2022] PUT /identity/v3/projects/34b8d99c4a214eeb99f7f839afffffd5/users/254d1ba2b4b140769de8785b9f3ddae6/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:21.484184 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08016595-c96f-466f-9c16-f52ddd71c9ac None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:21.484184 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08016595-c96f-466f-9c16-f52ddd71c9ac None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:21.484184 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08016595-c96f-466f-9c16-f52ddd71c9ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:21.499372 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:21.531198 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-08016595-c96f-466f-9c16-f52ddd71c9ac None None] MFA Rules not processed for user `53acb1b88afe49a7a90075826f46d62e`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:21.537347 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-08016595-c96f-466f-9c16-f52ddd71c9ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:21.568401 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50fff7a-dbd8-4e37-b032-5af3603b7a52 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:21.569942 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50fff7a-dbd8-4e37-b032-5af3603b7a52 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:21.593835 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 656/1311] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:33:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:33:21.596975 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef1593-f403-49b1-84d3-57d9a15554fd None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:21.597784 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef1593-f403-49b1-84d3-57d9a15554fd None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:21.599386 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ef1593-f403-49b1-84d3-57d9a15554fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:21.632904 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:21.665834 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-80ef1593-f403-49b1-84d3-57d9a15554fd None None] MFA Rules not processed for user `254d1ba2b4b140769de8785b9f3ddae6`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:21.667232 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-80ef1593-f403-49b1-84d3-57d9a15554fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:21.671801 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 656/1312] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:21 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:21.696177 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08016595-c96f-466f-9c16-f52ddd71c9ac None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:21.842903 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08016595-c96f-466f-9c16-f52ddd71c9ac None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:21.852557 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c93049b-adeb-49b9-bcd9-c2ab7e167d0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:21.853223 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93049b-adeb-49b9-bcd9-c2ab7e167d0c service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:21.853542 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93049b-adeb-49b9-bcd9-c2ab7e167d0c service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:21.853812 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c93049b-adeb-49b9-bcd9-c2ab7e167d0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:21.874842 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 657/1313] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:21 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:21.878514 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c93049b-adeb-49b9-bcd9-c2ab7e167d0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:21.880063 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c93049b-adeb-49b9-bcd9-c2ab7e167d0c service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:21.905074 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 657/1314] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:33:21 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:33:22.045459 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ef1593-f403-49b1-84d3-57d9a15554fd None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:22.055865 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ef1593-f403-49b1-84d3-57d9a15554fd None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:22.068662 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0e8830-0416-4e93-998c-2016032bf681 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:22.069529 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e8830-0416-4e93-998c-2016032bf681 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:22.069906 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e8830-0416-4e93-998c-2016032bf681 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:22.070272 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0e8830-0416-4e93-998c-2016032bf681 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:22.092270 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0e8830-0416-4e93-998c-2016032bf681 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:22.093893 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0e8830-0416-4e93-998c-2016032bf681 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:22.117407 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 658/1315] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:33:22 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:33:22.575213 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c93049b-adeb-49b9-bcd9-c2ab7e167d0c service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:22.759420 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c93049b-adeb-49b9-bcd9-c2ab7e167d0c service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:22.783606 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d434e-6587-4112-8b30-72005696646b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:22.786323 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d434e-6587-4112-8b30-72005696646b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:22.786846 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d434e-6587-4112-8b30-72005696646b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:22.787546 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d434e-6587-4112-8b30-72005696646b admin admin] PATH_INFO: `/v3/users/ce0536b28d3e457da55c4f4d1c618e4b` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:22.797046 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803d434e-6587-4112-8b30-72005696646b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce0536b28d3e457da55c4f4d1c618e4b)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:22.802013 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803d434e-6587-4112-8b30-72005696646b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:22.926937 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-803d434e-6587-4112-8b30-72005696646b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce0536b28d3e457da55c4f4d1c618e4b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:22.927897 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-803d434e-6587-4112-8b30-72005696646b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce0536b28d3e457da55c4f4d1c618e4b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:22.929346 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-803d434e-6587-4112-8b30-72005696646b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce0536b28d3e457da55c4f4d1c618e4b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:22.967102 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-803d434e-6587-4112-8b30-72005696646b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce0536b28d3e457da55c4f4d1c618e4b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:22.993915 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-803d434e-6587-4112-8b30-72005696646b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce0536b28d3e457da55c4f4d1c618e4b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:23.038896 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 658/1316] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 03:33:22 2022] DELETE /identity/v3/users/ce0536b28d3e457da55c4f4d1c618e4b => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:23.044618 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b27502d-be27-4ba2-8e74-9b2563955de6 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:23.045035 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b27502d-be27-4ba2-8e74-9b2563955de6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:23.045414 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b27502d-be27-4ba2-8e74-9b2563955de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:23.095283 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-3b27502d-be27-4ba2-8e74-9b2563955de6 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:23.096143 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3b27502d-be27-4ba2-8e74-9b2563955de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:23.241934 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 659/1317] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:23.270923 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d434e-6587-4112-8b30-72005696646b admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:23.277520 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-803d434e-6587-4112-8b30-72005696646b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:23.283665 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b27502d-be27-4ba2-8e74-9b2563955de6 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:23.285401 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3b27502d-be27-4ba2-8e74-9b2563955de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:23.313831 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d434e-6587-4112-8b30-72005696646b admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:23.328080 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d73787-2d92-4c85-a3a6-b765e25daefd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:23.328787 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d73787-2d92-4c85-a3a6-b765e25daefd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:23.330242 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d73787-2d92-4c85-a3a6-b765e25daefd admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:23.330519 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d73787-2d92-4c85-a3a6-b765e25daefd admin admin] PATH_INFO: `/v3/users/cf5c58f066ca4419ae81aa9732eda236` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:23.333189 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d73787-2d92-4c85-a3a6-b765e25daefd admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf5c58f066ca4419ae81aa9732eda236)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:23.336266 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d73787-2d92-4c85-a3a6-b765e25daefd admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:23.340437 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:23.370299 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:23.409516 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b27502d-be27-4ba2-8e74-9b2563955de6 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:23.413924 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0669cf-ce89-4658-943d-6d0e60bf4864 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:23.414650 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0669cf-ce89-4658-943d-6d0e60bf4864 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:23.414972 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0669cf-ce89-4658-943d-6d0e60bf4864 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:23.415252 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0669cf-ce89-4658-943d-6d0e60bf4864 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:23.429734 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0669cf-ce89-4658-943d-6d0e60bf4864 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:23.442530 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0669cf-ce89-4658-943d-6d0e60bf4864 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:23.443543 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-51d73787-2d92-4c85-a3a6-b765e25daefd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf5c58f066ca4419ae81aa9732eda236'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:23.444362 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-51d73787-2d92-4c85-a3a6-b765e25daefd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf5c58f066ca4419ae81aa9732eda236'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:23.444995 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-51d73787-2d92-4c85-a3a6-b765e25daefd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf5c58f066ca4419ae81aa9732eda236'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:23.462812 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 660/1318] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:33:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:33:23.464665 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef5510c-4b8a-47ae-a1bb-6fc993aa4874 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:23.465102 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef5510c-4b8a-47ae-a1bb-6fc993aa4874 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:23.465526 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef5510c-4b8a-47ae-a1bb-6fc993aa4874 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:23.467584 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 661/1319] 10.222.0.26 () {54 vars in 961 bytes} [Wed Nov 2 03:33:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:33:23.476394 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-51d73787-2d92-4c85-a3a6-b765e25daefd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf5c58f066ca4419ae81aa9732eda236'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:23.478175 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2fc3e3-478b-4b15-9272-387e2b65d646 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:23.478677 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2fc3e3-478b-4b15-9272-387e2b65d646 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:23.479448 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2fc3e3-478b-4b15-9272-387e2b65d646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:23.492290 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-51d73787-2d92-4c85-a3a6-b765e25daefd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf5c58f066ca4419ae81aa9732eda236'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:23.513718 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-df2fc3e3-478b-4b15-9272-387e2b65d646 None None] MFA Rules not processed for user `44961b06151c4da382fcd54dd1f1e146`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:23.515545 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-df2fc3e3-478b-4b15-9272-387e2b65d646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:23.529529 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 659/1320] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 03:33:23 2022] DELETE /identity/v3/users/cf5c58f066ca4419ae81aa9732eda236 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:23.531276 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e8f67-60d1-45a1-835a-2e5ea5f69b01 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:23.531495 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e8f67-60d1-45a1-835a-2e5ea5f69b01 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:23.531838 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e8f67-60d1-45a1-835a-2e5ea5f69b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:23.593202 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-364e8f67-60d1-45a1-835a-2e5ea5f69b01 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:23.596137 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-364e8f67-60d1-45a1-835a-2e5ea5f69b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:23.697679 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 662/1321] 10.222.0.26 () {58 vars in 1058 bytes} [Wed Nov 2 03:33:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:23.702835 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32aa91b-4aaf-4f1f-a493-1576c141c86f None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:23.703576 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32aa91b-4aaf-4f1f-a493-1576c141c86f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:23.706832 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32aa91b-4aaf-4f1f-a493-1576c141c86f None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:23.716357 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 663/1322] 10.222.0.26 () {54 vars in 912 bytes} [Wed Nov 2 03:33:23 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:33:23.722695 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df44d1df-5bca-4e22-96db-2501138596d4 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:23.722961 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df44d1df-5bca-4e22-96db-2501138596d4 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:23.723242 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df44d1df-5bca-4e22-96db-2501138596d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:23.734101 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 660/1323] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:23.751795 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-df44d1df-5bca-4e22-96db-2501138596d4 None None] MFA Rules not processed for user `44961b06151c4da382fcd54dd1f1e146`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:23.753198 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-df44d1df-5bca-4e22-96db-2501138596d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:23.764613 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 664/1324] 10.222.0.26 () {58 vars in 1009 bytes} [Wed Nov 2 03:33:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 03:33:23.778352 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364e8f67-60d1-45a1-835a-2e5ea5f69b01 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:23.779887 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-364e8f67-60d1-45a1-835a-2e5ea5f69b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:23.851921 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df44d1df-5bca-4e22-96db-2501138596d4 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:23.853603 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-df44d1df-5bca-4e22-96db-2501138596d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:23.872610 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364e8f67-60d1-45a1-835a-2e5ea5f69b01 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:23.886419 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527ad3d3-5d46-4264-bbd3-08ad371c87b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:23.887754 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ad3d3-5d46-4264-bbd3-08ad371c87b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:23.888036 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ad3d3-5d46-4264-bbd3-08ad371c87b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:23.888221 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ad3d3-5d46-4264-bbd3-08ad371c87b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:23.919311 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527ad3d3-5d46-4264-bbd3-08ad371c87b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:23.921646 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527ad3d3-5d46-4264-bbd3-08ad371c87b5 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:23.952887 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 661/1325] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:33:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:33:23.970595 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df44d1df-5bca-4e22-96db-2501138596d4 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:23.980800 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67adbd52-906b-4295-a511-2f736aa93c3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:23.981706 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67adbd52-906b-4295-a511-2f736aa93c3b service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:23.982135 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67adbd52-906b-4295-a511-2f736aa93c3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:23.983266 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67adbd52-906b-4295-a511-2f736aa93c3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:23.996676 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67adbd52-906b-4295-a511-2f736aa93c3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:23.998713 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67adbd52-906b-4295-a511-2f736aa93c3b service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:24.010476 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 665/1326] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:33:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 03:33:24.858891 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527ad3d3-5d46-4264-bbd3-08ad371c87b5 service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:24.862603 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-527ad3d3-5d46-4264-bbd3-08ad371c87b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:24.884581 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527ad3d3-5d46-4264-bbd3-08ad371c87b5 service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:24.890970 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a93a886-cfb5-4966-8826-62e86080909e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:24.891918 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93a886-cfb5-4966-8826-62e86080909e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:24.892687 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93a886-cfb5-4966-8826-62e86080909e admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:24.893160 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a93a886-cfb5-4966-8826-62e86080909e admin admin] PATH_INFO: `/v3/projects/8811221b05f8451f8448997951fdf35f` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:24.894515 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a93a886-cfb5-4966-8826-62e86080909e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8811221b05f8451f8448997951fdf35f)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:24.895784 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: WARNING py.warnings [None req-6a93a886-cfb5-4966-8826-62e86080909e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 03:33:24.895784 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: warnings.warn(msg) Nov 02 03:33:24.895784 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: Nov 02 03:33:24.897555 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a93a886-cfb5-4966-8826-62e86080909e admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:24.955673 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:25.011248 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-6a93a886-cfb5-4966-8826-62e86080909e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8811221b05f8451f8448997951fdf35f', 'request_id': 'req-6a93a886-cfb5-4966-8826-62e86080909e'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:25.021254 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-6a93a886-cfb5-4966-8826-62e86080909e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8811221b05f8451f8448997951fdf35f', 'request_id': 'req-6a93a886-cfb5-4966-8826-62e86080909e'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:25.046121 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-6a93a886-cfb5-4966-8826-62e86080909e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8811221b05f8451f8448997951fdf35f', 'request_id': 'req-6a93a886-cfb5-4966-8826-62e86080909e'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:25.065945 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-6a93a886-cfb5-4966-8826-62e86080909e admin admin] The token cache is being invalidate because project 8811221b05f8451f8448997951fdf35f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 03:33:25.066431 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-6a93a886-cfb5-4966-8826-62e86080909e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:25.067241 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-6a93a886-cfb5-4966-8826-62e86080909e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:25.072560 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 662/1327] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 03:33:24 2022] DELETE /identity/v3/projects/8811221b05f8451f8448997951fdf35f => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:25.073395 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67adbd52-906b-4295-a511-2f736aa93c3b service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:25.085982 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67adbd52-906b-4295-a511-2f736aa93c3b service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:25.223557 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81a41c3-2416-4275-a036-bacd67225e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:25.224599 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81a41c3-2416-4275-a036-bacd67225e71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:25.225082 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81a41c3-2416-4275-a036-bacd67225e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:25.225534 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81a41c3-2416-4275-a036-bacd67225e71 admin admin] PATH_INFO: `/v3/users/53acb1b88afe49a7a90075826f46d62e` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:25.228309 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81a41c3-2416-4275-a036-bacd67225e71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=53acb1b88afe49a7a90075826f46d62e)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:25.230136 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: WARNING py.warnings [None req-e81a41c3-2416-4275-a036-bacd67225e71 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Nov 02 03:33:25.230136 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: warnings.warn(msg) Nov 02 03:33:25.230136 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: Nov 02 03:33:25.234434 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81a41c3-2416-4275-a036-bacd67225e71 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:25.242583 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:25.351351 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e81a41c3-2416-4275-a036-bacd67225e71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53acb1b88afe49a7a90075826f46d62e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:25.369784 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e81a41c3-2416-4275-a036-bacd67225e71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53acb1b88afe49a7a90075826f46d62e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:25.370981 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e81a41c3-2416-4275-a036-bacd67225e71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53acb1b88afe49a7a90075826f46d62e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:25.405678 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e81a41c3-2416-4275-a036-bacd67225e71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53acb1b88afe49a7a90075826f46d62e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:25.416604 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e81a41c3-2416-4275-a036-bacd67225e71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53acb1b88afe49a7a90075826f46d62e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:25.427971 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 666/1328] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 03:33:25 2022] DELETE /identity/v3/users/53acb1b88afe49a7a90075826f46d62e => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:26.281436 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a93a886-cfb5-4966-8826-62e86080909e admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:26.282980 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-6a93a886-cfb5-4966-8826-62e86080909e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:26.306770 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a93a886-cfb5-4966-8826-62e86080909e admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:26.321027 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81a41c3-2416-4275-a036-bacd67225e71 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:26.322680 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e81a41c3-2416-4275-a036-bacd67225e71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:26.437846 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4fd8f3-8108-48ce-8afe-a685b48f0165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:26.443494 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4fd8f3-8108-48ce-8afe-a685b48f0165 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:26.444107 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4fd8f3-8108-48ce-8afe-a685b48f0165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:26.444660 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4fd8f3-8108-48ce-8afe-a685b48f0165 admin admin] PATH_INFO: `/v3/projects/e834781d69d64f57a5da8e634428c0fb` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:26.446382 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4fd8f3-8108-48ce-8afe-a685b48f0165 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e834781d69d64f57a5da8e634428c0fb)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:26.450828 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4fd8f3-8108-48ce-8afe-a685b48f0165 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:26.553544 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81a41c3-2416-4275-a036-bacd67225e71 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:26.584678 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-bc4fd8f3-8108-48ce-8afe-a685b48f0165 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e834781d69d64f57a5da8e634428c0fb', 'request_id': 'req-bc4fd8f3-8108-48ce-8afe-a685b48f0165'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:26.603040 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-bc4fd8f3-8108-48ce-8afe-a685b48f0165 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e834781d69d64f57a5da8e634428c0fb', 'request_id': 'req-bc4fd8f3-8108-48ce-8afe-a685b48f0165'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:26.610615 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-bc4fd8f3-8108-48ce-8afe-a685b48f0165 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e834781d69d64f57a5da8e634428c0fb', 'request_id': 'req-bc4fd8f3-8108-48ce-8afe-a685b48f0165'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:26.632020 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-bc4fd8f3-8108-48ce-8afe-a685b48f0165 admin admin] The token cache is being invalidate because project e834781d69d64f57a5da8e634428c0fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 03:33:26.632591 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-bc4fd8f3-8108-48ce-8afe-a685b48f0165 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:26.633904 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-bc4fd8f3-8108-48ce-8afe-a685b48f0165 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:26.635877 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 663/1329] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 03:33:26 2022] DELETE /identity/v3/projects/e834781d69d64f57a5da8e634428c0fb => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:26.639990 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4fd8f3-8108-48ce-8afe-a685b48f0165 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:26.641618 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-bc4fd8f3-8108-48ce-8afe-a685b48f0165 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:26.734440 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:26.769704 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed14d12-6aa9-4c9e-83cb-1a51205428f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:26.770681 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed14d12-6aa9-4c9e-83cb-1a51205428f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:26.771044 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed14d12-6aa9-4c9e-83cb-1a51205428f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:26.771396 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed14d12-6aa9-4c9e-83cb-1a51205428f9 admin admin] PATH_INFO: `/v3/projects/0b8c8e9a2d0c4d9c93aa6636ecd72557` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:26.774208 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed14d12-6aa9-4c9e-83cb-1a51205428f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b8c8e9a2d0c4d9c93aa6636ecd72557)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:26.776444 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed14d12-6aa9-4c9e-83cb-1a51205428f9 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:26.867369 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4fd8f3-8108-48ce-8afe-a685b48f0165 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:26.885806 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:26.892413 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae9307d-3ac3-4873-a26b-1b94db3d90c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:26.893166 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae9307d-3ac3-4873-a26b-1b94db3d90c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:26.893686 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae9307d-3ac3-4873-a26b-1b94db3d90c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:26.894071 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae9307d-3ac3-4873-a26b-1b94db3d90c3 admin admin] PATH_INFO: `/v3/users/254d1ba2b4b140769de8785b9f3ddae6` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:26.897202 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae9307d-3ac3-4873-a26b-1b94db3d90c3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=254d1ba2b4b140769de8785b9f3ddae6)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:26.902337 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae9307d-3ac3-4873-a26b-1b94db3d90c3 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:26.928030 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4ed14d12-6aa9-4c9e-83cb-1a51205428f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b8c8e9a2d0c4d9c93aa6636ecd72557', 'request_id': 'req-4ed14d12-6aa9-4c9e-83cb-1a51205428f9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:26.946690 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4ed14d12-6aa9-4c9e-83cb-1a51205428f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b8c8e9a2d0c4d9c93aa6636ecd72557', 'request_id': 'req-4ed14d12-6aa9-4c9e-83cb-1a51205428f9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:26.982850 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4ed14d12-6aa9-4c9e-83cb-1a51205428f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b8c8e9a2d0c4d9c93aa6636ecd72557', 'request_id': 'req-4ed14d12-6aa9-4c9e-83cb-1a51205428f9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:27.016284 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4ed14d12-6aa9-4c9e-83cb-1a51205428f9 admin admin] The token cache is being invalidate because project 0b8c8e9a2d0c4d9c93aa6636ecd72557 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 03:33:27.017293 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4ed14d12-6aa9-4c9e-83cb-1a51205428f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:27.019856 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4ed14d12-6aa9-4c9e-83cb-1a51205428f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:27.040449 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 667/1330] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 03:33:26 2022] DELETE /identity/v3/projects/0b8c8e9a2d0c4d9c93aa6636ecd72557 => generated 0 bytes in 715 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:27.044114 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdeee16-3c62-4d04-93d4-963726a08bda None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:27.044114 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdeee16-3c62-4d04-93d4-963726a08bda None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:27.053017 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdeee16-3c62-4d04-93d4-963726a08bda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:27.065850 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5ae9307d-3ac3-4873-a26b-1b94db3d90c3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '254d1ba2b4b140769de8785b9f3ddae6'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:27.067927 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5ae9307d-3ac3-4873-a26b-1b94db3d90c3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '254d1ba2b4b140769de8785b9f3ddae6'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:27.073779 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5ae9307d-3ac3-4873-a26b-1b94db3d90c3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '254d1ba2b4b140769de8785b9f3ddae6'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:27.108689 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-7cdeee16-3c62-4d04-93d4-963726a08bda None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:27.110554 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7cdeee16-3c62-4d04-93d4-963726a08bda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:27.119119 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5ae9307d-3ac3-4873-a26b-1b94db3d90c3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '254d1ba2b4b140769de8785b9f3ddae6'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:27.137218 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5ae9307d-3ac3-4873-a26b-1b94db3d90c3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '254d1ba2b4b140769de8785b9f3ddae6'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:27.178668 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 664/1331] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 03:33:26 2022] DELETE /identity/v3/users/254d1ba2b4b140769de8785b9f3ddae6 => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:27.387855 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 668/1332] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:27.397679 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002c377c-8c94-419b-97a5-dc1978265a82 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:27.398532 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002c377c-8c94-419b-97a5-dc1978265a82 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:27.398955 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002c377c-8c94-419b-97a5-dc1978265a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:27.460978 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-002c377c-8c94-419b-97a5-dc1978265a82 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:27.462205 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-002c377c-8c94-419b-97a5-dc1978265a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:27.728272 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 665/1333] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:27.729803 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdeee16-3c62-4d04-93d4-963726a08bda None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:27.743241 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdeee16-3c62-4d04-93d4-963726a08bda None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:27.769896 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b92adf5-7fda-46f0-936c-698365ba8dcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:27.770664 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b92adf5-7fda-46f0-936c-698365ba8dcb admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:27.770854 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b92adf5-7fda-46f0-936c-698365ba8dcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:27.771132 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b92adf5-7fda-46f0-936c-698365ba8dcb admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:27.771793 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b92adf5-7fda-46f0-936c-698365ba8dcb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:33:27.772128 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b92adf5-7fda-46f0-936c-698365ba8dcb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:27.776055 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b92adf5-7fda-46f0-936c-698365ba8dcb admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:27.843674 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 669/1334] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 03:33:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:33:27.845979 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002c377c-8c94-419b-97a5-dc1978265a82 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:27.859372 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002c377c-8c94-419b-97a5-dc1978265a82 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:27.875238 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77997e60-e3db-4805-9913-2bf61877776e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:27.875896 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77997e60-e3db-4805-9913-2bf61877776e admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:27.876258 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77997e60-e3db-4805-9913-2bf61877776e admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:27.876540 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77997e60-e3db-4805-9913-2bf61877776e admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:27.877209 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77997e60-e3db-4805-9913-2bf61877776e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:27.879129 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77997e60-e3db-4805-9913-2bf61877776e admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:27.955066 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 666/1335] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:33:27 2022] POST /identity/v3/projects => generated 369 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:33:27.959338 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b92adf5-7fda-46f0-936c-698365ba8dcb admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:27.973754 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b92adf5-7fda-46f0-936c-698365ba8dcb admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:28.104474 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af496ede-3568-49df-9961-9c3b0a910121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:28.105275 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af496ede-3568-49df-9961-9c3b0a910121 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:28.105764 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af496ede-3568-49df-9961-9c3b0a910121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:28.106147 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af496ede-3568-49df-9961-9c3b0a910121 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:28.107037 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af496ede-3568-49df-9961-9c3b0a910121 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:28.109280 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af496ede-3568-49df-9961-9c3b0a910121 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:28.148787 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77997e60-e3db-4805-9913-2bf61877776e admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:28.151282 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 670/1336] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:33:27 2022] POST /identity/v3/users => generated 285 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:33:28.165914 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77997e60-e3db-4805-9913-2bf61877776e admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:28.173854 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee2ed5b-bbeb-4499-946e-9e5bc2f0749c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:28.174733 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee2ed5b-bbeb-4499-946e-9e5bc2f0749c admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:28.175169 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee2ed5b-bbeb-4499-946e-9e5bc2f0749c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:28.175550 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee2ed5b-bbeb-4499-946e-9e5bc2f0749c admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:28.176674 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee2ed5b-bbeb-4499-946e-9e5bc2f0749c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:28.178520 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee2ed5b-bbeb-4499-946e-9e5bc2f0749c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:28.198208 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af496ede-3568-49df-9961-9c3b0a910121 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:28.201834 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 667/1337] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:33:28 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:33:28.222778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af496ede-3568-49df-9961-9c3b0a910121 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:28.244642 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8f1737-997e-4090-8b9a-fcda631e6c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:28.246133 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f1737-997e-4090-8b9a-fcda631e6c7f admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:28.248222 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f1737-997e-4090-8b9a-fcda631e6c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:28.248745 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8f1737-997e-4090-8b9a-fcda631e6c7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:28.254288 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8f1737-997e-4090-8b9a-fcda631e6c7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:28.255978 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8f1737-997e-4090-8b9a-fcda631e6c7f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:28.292496 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 671/1338] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:33:28 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:33:28.294155 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee2ed5b-bbeb-4499-946e-9e5bc2f0749c admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:28.309630 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee2ed5b-bbeb-4499-946e-9e5bc2f0749c admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:28.327202 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaf7325-8a96-47b9-95af-db0376c63d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:28.327905 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf7325-8a96-47b9-95af-db0376c63d7f admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:28.328319 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf7325-8a96-47b9-95af-db0376c63d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:28.328507 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfaf7325-8a96-47b9-95af-db0376c63d7f admin admin] PATH_INFO: `/v3/projects/73435b08d77c4f65a36325526a2baf29/users/082f61c1659441179e1e1b22f8e75966/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:28.368627 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaf7325-8a96-47b9-95af-db0376c63d7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=73435b08d77c4f65a36325526a2baf29, user_id=082f61c1659441179e1e1b22f8e75966, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:28.380889 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfaf7325-8a96-47b9-95af-db0376c63d7f admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:28.414013 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:28.430482 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 668/1339] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:33:28 2022] PUT /identity/v3/projects/73435b08d77c4f65a36325526a2baf29/users/082f61c1659441179e1e1b22f8e75966/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:28.441142 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a4af9-8480-499e-a779-d24d40e26b62 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:28.441679 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a4af9-8480-499e-a779-d24d40e26b62 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:28.441679 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a4af9-8480-499e-a779-d24d40e26b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:28.549439 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-391a4af9-8480-499e-a779-d24d40e26b62 None None] MFA Rules not processed for user `082f61c1659441179e1e1b22f8e75966`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:28.550692 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-391a4af9-8480-499e-a779-d24d40e26b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:28.592677 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:28.726062 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 672/1340] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:28 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:28.730222 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaf7325-8a96-47b9-95af-db0376c63d7f admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:28.883683 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfaf7325-8a96-47b9-95af-db0376c63d7f admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:28.900317 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a5865-f511-46f3-a488-1c171b91fb14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:28.901078 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a5865-f511-46f3-a488-1c171b91fb14 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:28.901461 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a5865-f511-46f3-a488-1c171b91fb14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:28.901845 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907a5865-f511-46f3-a488-1c171b91fb14 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:28.902615 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907a5865-f511-46f3-a488-1c171b91fb14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:28.904721 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907a5865-f511-46f3-a488-1c171b91fb14 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:28.977053 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 669/1341] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:33:28 2022] POST /identity/v3/projects => generated 369 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:33:28.983491 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391a4af9-8480-499e-a779-d24d40e26b62 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:29.018106 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391a4af9-8480-499e-a779-d24d40e26b62 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:29.168172 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b93739-6ef4-4181-9b21-284d987b8549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:29.169105 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b93739-6ef4-4181-9b21-284d987b8549 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:29.169559 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b93739-6ef4-4181-9b21-284d987b8549 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:29.170061 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b93739-6ef4-4181-9b21-284d987b8549 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:29.174152 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b93739-6ef4-4181-9b21-284d987b8549 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:29.177537 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b93739-6ef4-4181-9b21-284d987b8549 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:29.183517 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a5865-f511-46f3-a488-1c171b91fb14 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:29.185366 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-907a5865-f511-46f3-a488-1c171b91fb14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:29.219051 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907a5865-f511-46f3-a488-1c171b91fb14 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:29.244583 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2853b1-205e-4fa6-ac70-59eb7602e242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:29.245373 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2853b1-205e-4fa6-ac70-59eb7602e242 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:29.245657 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2853b1-205e-4fa6-ac70-59eb7602e242 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:29.245934 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2853b1-205e-4fa6-ac70-59eb7602e242 admin admin] PATH_INFO: `/v3/projects/34b8d99c4a214eeb99f7f839afffffd5` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:29.248990 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2853b1-205e-4fa6-ac70-59eb7602e242 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34b8d99c4a214eeb99f7f839afffffd5)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:29.250669 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2853b1-205e-4fa6-ac70-59eb7602e242 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:29.254294 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 673/1342] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:33:28 2022] POST /identity/v3/users => generated 291 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:33:29.263250 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b93739-6ef4-4181-9b21-284d987b8549 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:29.310500 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b93739-6ef4-4181-9b21-284d987b8549 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:29.333422 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe81b1-4a60-4d29-8b5b-77bca1f1604c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:29.336209 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe81b1-4a60-4d29-8b5b-77bca1f1604c admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:29.336764 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe81b1-4a60-4d29-8b5b-77bca1f1604c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:29.337563 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefe81b1-4a60-4d29-8b5b-77bca1f1604c admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:29.345712 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefe81b1-4a60-4d29-8b5b-77bca1f1604c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:29.349308 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cefe81b1-4a60-4d29-8b5b-77bca1f1604c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:29.387485 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 674/1343] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:33:29 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:33:29.391442 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe81b1-4a60-4d29-8b5b-77bca1f1604c admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:29.408953 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-6d2853b1-205e-4fa6-ac70-59eb7602e242 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34b8d99c4a214eeb99f7f839afffffd5', 'request_id': 'req-6d2853b1-205e-4fa6-ac70-59eb7602e242'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:29.424548 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-6d2853b1-205e-4fa6-ac70-59eb7602e242 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34b8d99c4a214eeb99f7f839afffffd5', 'request_id': 'req-6d2853b1-205e-4fa6-ac70-59eb7602e242'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:29.432330 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cefe81b1-4a60-4d29-8b5b-77bca1f1604c admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:29.452502 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-6d2853b1-205e-4fa6-ac70-59eb7602e242 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34b8d99c4a214eeb99f7f839afffffd5', 'request_id': 'req-6d2853b1-205e-4fa6-ac70-59eb7602e242'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:29.479835 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-6d2853b1-205e-4fa6-ac70-59eb7602e242 admin admin] The token cache is being invalidate because project 34b8d99c4a214eeb99f7f839afffffd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 03:33:29.480211 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-6d2853b1-205e-4fa6-ac70-59eb7602e242 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:29.481752 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-6d2853b1-205e-4fa6-ac70-59eb7602e242 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:29.486194 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 670/1344] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 03:33:29 2022] DELETE /identity/v3/projects/34b8d99c4a214eeb99f7f839afffffd5 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:29.494181 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e572fecd-9edc-4266-8bfb-90f73704b6bd None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:29.494616 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e572fecd-9edc-4266-8bfb-90f73704b6bd None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:29.495165 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e572fecd-9edc-4266-8bfb-90f73704b6bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:29.560758 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-e572fecd-9edc-4266-8bfb-90f73704b6bd None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:29.565146 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e572fecd-9edc-4266-8bfb-90f73704b6bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:29.660469 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb69a53-6330-43d1-a660-05861b951005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:29.661313 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb69a53-6330-43d1-a660-05861b951005 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:29.661707 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb69a53-6330-43d1-a660-05861b951005 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:29.662091 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb69a53-6330-43d1-a660-05861b951005 admin admin] PATH_INFO: `/v3/projects/158f6689158d475c96e7aa783468accc/users/894a90c7c5ac4e5f98415f2a12645e69/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:29.705459 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb69a53-6330-43d1-a660-05861b951005 admin admin] RBAC: Authorizing `identity:create_grant(project_id=158f6689158d475c96e7aa783468accc, user_id=894a90c7c5ac4e5f98415f2a12645e69, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:29.708657 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb69a53-6330-43d1-a660-05861b951005 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:29.744893 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 675/1345] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:33:29 2022] PUT /identity/v3/projects/158f6689158d475c96e7aa783468accc/users/894a90c7c5ac4e5f98415f2a12645e69/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:29.752316 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb69a53-6330-43d1-a660-05861b951005 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:29.757813 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8bb69a53-6330-43d1-a660-05861b951005 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:29.804545 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:29.824679 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 671/1346] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:29.834692 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37ced2-5ede-463f-8eca-b3430f4d86a7 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:29.834980 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37ced2-5ede-463f-8eca-b3430f4d86a7 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:29.835262 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f37ced2-5ede-463f-8eca-b3430f4d86a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:29.893401 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-1f37ced2-5ede-463f-8eca-b3430f4d86a7 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:29.898323 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-1f37ced2-5ede-463f-8eca-b3430f4d86a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:29.920655 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:29.958964 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb69a53-6330-43d1-a660-05861b951005 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:29.982157 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4626b43e-514b-4ff2-98ae-e0ce05e5e3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:29.983729 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4626b43e-514b-4ff2-98ae-e0ce05e5e3d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:29.984049 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4626b43e-514b-4ff2-98ae-e0ce05e5e3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:29.984331 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4626b43e-514b-4ff2-98ae-e0ce05e5e3d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:29.984993 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4626b43e-514b-4ff2-98ae-e0ce05e5e3d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:29.987191 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4626b43e-514b-4ff2-98ae-e0ce05e5e3d1 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:30.027355 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 676/1347] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:33:29 2022] GET /identity/v3/roles => generated 1420 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:33:30.028302 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4626b43e-514b-4ff2-98ae-e0ce05e5e3d1 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:30.051328 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4626b43e-514b-4ff2-98ae-e0ce05e5e3d1 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:30.075242 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99432809-2e73-4fe8-91ac-709f6950ee04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:30.076721 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99432809-2e73-4fe8-91ac-709f6950ee04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:30.081854 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99432809-2e73-4fe8-91ac-709f6950ee04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:30.082804 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99432809-2e73-4fe8-91ac-709f6950ee04 admin admin] PATH_INFO: `/v3/projects/158f6689158d475c96e7aa783468accc/users/894a90c7c5ac4e5f98415f2a12645e69/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:30.088196 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99432809-2e73-4fe8-91ac-709f6950ee04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=158f6689158d475c96e7aa783468accc, user_id=894a90c7c5ac4e5f98415f2a12645e69, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:30.101254 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99432809-2e73-4fe8-91ac-709f6950ee04 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:30.108553 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 672/1348] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 03:33:30.115838 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f37ced2-5ede-463f-8eca-b3430f4d86a7 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:30.154584 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 677/1349] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:33:30 2022] PUT /identity/v3/projects/158f6689158d475c96e7aa783468accc/users/894a90c7c5ac4e5f98415f2a12645e69/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:30.165361 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ff56e-4dc2-4280-a737-539d332b35de None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:30.165717 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ff56e-4dc2-4280-a737-539d332b35de None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:30.165923 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27ff56e-4dc2-4280-a737-539d332b35de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:30.169707 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f37ced2-5ede-463f-8eca-b3430f4d86a7 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:30.258298 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d27ff56e-4dc2-4280-a737-539d332b35de None None] MFA Rules not processed for user `894a90c7c5ac4e5f98415f2a12645e69`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:30.267015 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d27ff56e-4dc2-4280-a737-539d332b35de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:30.294948 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b36391-1774-4701-af3d-23242895ef5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:30.295631 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b36391-1774-4701-af3d-23242895ef5a admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:30.295910 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b36391-1774-4701-af3d-23242895ef5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:30.296367 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b36391-1774-4701-af3d-23242895ef5a admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:30.297035 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b36391-1774-4701-af3d-23242895ef5a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:33:30.297332 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b36391-1774-4701-af3d-23242895ef5a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:30.299324 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b36391-1774-4701-af3d-23242895ef5a admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:30.330670 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 673/1350] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 03:33:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:33:30.332942 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b36391-1774-4701-af3d-23242895ef5a admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:30.346877 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b36391-1774-4701-af3d-23242895ef5a admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:30.353315 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5835dcc9-0995-4bf6-a0b3-c47a383b8271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:30.353937 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5835dcc9-0995-4bf6-a0b3-c47a383b8271 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:30.354217 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5835dcc9-0995-4bf6-a0b3-c47a383b8271 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:30.356159 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5835dcc9-0995-4bf6-a0b3-c47a383b8271 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:30.356819 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5835dcc9-0995-4bf6-a0b3-c47a383b8271 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:30.361364 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5835dcc9-0995-4bf6-a0b3-c47a383b8271 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:30.419541 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 678/1351] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:30 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:30.431348 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27ff56e-4dc2-4280-a737-539d332b35de None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:30.432788 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d27ff56e-4dc2-4280-a737-539d332b35de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:30.433709 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 674/1352] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:33:30 2022] POST /identity/v3/projects => generated 367 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:33:30.435766 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5835dcc9-0995-4bf6-a0b3-c47a383b8271 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:30.557107 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27ff56e-4dc2-4280-a737-539d332b35de None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:30.603495 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5835dcc9-0995-4bf6-a0b3-c47a383b8271 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:30.617277 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f306977-9d87-4c7a-93cb-c0bafa468b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:30.618037 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f306977-9d87-4c7a-93cb-c0bafa468b77 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:30.618450 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f306977-9d87-4c7a-93cb-c0bafa468b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:30.618954 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f306977-9d87-4c7a-93cb-c0bafa468b77 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:30.619730 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f306977-9d87-4c7a-93cb-c0bafa468b77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:30.624100 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f306977-9d87-4c7a-93cb-c0bafa468b77 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:30.657624 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:30.671293 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbce1f80-6744-457a-a7dd-cc9cb34599ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:30.671868 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce1f80-6744-457a-a7dd-cc9cb34599ab service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:30.672081 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce1f80-6744-457a-a7dd-cc9cb34599ab service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:30.672306 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce1f80-6744-457a-a7dd-cc9cb34599ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:30.679043 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 675/1353] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:33:30 2022] POST /identity/v3/users => generated 284 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:33:30.680152 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f306977-9d87-4c7a-93cb-c0bafa468b77 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:30.700852 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f306977-9d87-4c7a-93cb-c0bafa468b77 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:30.711358 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75d4b83-1bde-471c-9306-c4d3cc3e6ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:30.712207 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d4b83-1bde-471c-9306-c4d3cc3e6ee3 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:30.712595 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d4b83-1bde-471c-9306-c4d3cc3e6ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:30.712995 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d4b83-1bde-471c-9306-c4d3cc3e6ee3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:30.713863 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75d4b83-1bde-471c-9306-c4d3cc3e6ee3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:30.715572 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75d4b83-1bde-471c-9306-c4d3cc3e6ee3 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:30.723366 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:30.732514 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 676/1354] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:33:30 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:33:30.733866 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75d4b83-1bde-471c-9306-c4d3cc3e6ee3 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:30.750536 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75d4b83-1bde-471c-9306-c4d3cc3e6ee3 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:30.767552 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e310d468-e4b7-4e7c-b018-971bbc13c69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:30.768273 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e310d468-e4b7-4e7c-b018-971bbc13c69c admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:30.768822 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e310d468-e4b7-4e7c-b018-971bbc13c69c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:30.769571 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e310d468-e4b7-4e7c-b018-971bbc13c69c admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:30.770094 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e310d468-e4b7-4e7c-b018-971bbc13c69c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:30.777802 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e310d468-e4b7-4e7c-b018-971bbc13c69c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:30.806377 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 677/1355] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:33:30 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:33:30.807378 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e310d468-e4b7-4e7c-b018-971bbc13c69c admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:30.815320 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbce1f80-6744-457a-a7dd-cc9cb34599ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:30.817228 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbce1f80-6744-457a-a7dd-cc9cb34599ab service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:30.831314 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e310d468-e4b7-4e7c-b018-971bbc13c69c admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:30.840943 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 679/1356] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:33:30 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 03:33:30.852296 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41965463-e95b-45a7-abf5-f7da62bf30ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:30.856893 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41965463-e95b-45a7-abf5-f7da62bf30ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:30.857434 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41965463-e95b-45a7-abf5-f7da62bf30ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:30.857936 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41965463-e95b-45a7-abf5-f7da62bf30ed admin admin] PATH_INFO: `/v3/projects/94822d694220434bb08773106f1cb157/users/8d9b08ab10f64ccea3015a7570ab0b84/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:30.906157 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41965463-e95b-45a7-abf5-f7da62bf30ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=94822d694220434bb08773106f1cb157, user_id=8d9b08ab10f64ccea3015a7570ab0b84, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:30.909514 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41965463-e95b-45a7-abf5-f7da62bf30ed admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:30.956400 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 678/1357] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:33:30 2022] PUT /identity/v3/projects/94822d694220434bb08773106f1cb157/users/8d9b08ab10f64ccea3015a7570ab0b84/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:30.967333 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05844b9f-8ebe-43a3-a860-80f577338f8f None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:30.967602 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05844b9f-8ebe-43a3-a860-80f577338f8f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:30.967876 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05844b9f-8ebe-43a3-a860-80f577338f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:31.044804 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-05844b9f-8ebe-43a3-a860-80f577338f8f None None] MFA Rules not processed for user `8d9b08ab10f64ccea3015a7570ab0b84`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:31.046068 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-05844b9f-8ebe-43a3-a860-80f577338f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:31.209504 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 680/1358] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:30 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:31.210790 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41965463-e95b-45a7-abf5-f7da62bf30ed admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:31.519295 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41965463-e95b-45a7-abf5-f7da62bf30ed admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:31.547618 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a9c3f1-54ab-4cd9-b004-82418941662c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:31.548860 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9c3f1-54ab-4cd9-b004-82418941662c admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:31.550281 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9c3f1-54ab-4cd9-b004-82418941662c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:31.551244 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9c3f1-54ab-4cd9-b004-82418941662c admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:31.551977 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a9c3f1-54ab-4cd9-b004-82418941662c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:31.553908 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a9c3f1-54ab-4cd9-b004-82418941662c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:31.606889 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05844b9f-8ebe-43a3-a860-80f577338f8f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:31.608298 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-05844b9f-8ebe-43a3-a860-80f577338f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:31.640199 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 679/1359] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:33:31 2022] POST /identity/v3/projects => generated 365 bytes in 430 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:33:31.644695 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a9c3f1-54ab-4cd9-b004-82418941662c admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:31.685168 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05844b9f-8ebe-43a3-a860-80f577338f8f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:31.862629 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a9c3f1-54ab-4cd9-b004-82418941662c admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:31.890775 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e9440e-0a08-43b5-b875-223e6a0a0aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:31.891939 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e9440e-0a08-43b5-b875-223e6a0a0aac admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:31.891939 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e9440e-0a08-43b5-b875-223e6a0a0aac admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:31.895124 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e9440e-0a08-43b5-b875-223e6a0a0aac admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:31.896667 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e9440e-0a08-43b5-b875-223e6a0a0aac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:31.899572 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e9440e-0a08-43b5-b875-223e6a0a0aac admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:31.948477 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e932f423-56fc-4b9b-b2c4-8d3bb3caa58f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:31.949304 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e932f423-56fc-4b9b-b2c4-8d3bb3caa58f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:31.949405 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e932f423-56fc-4b9b-b2c4-8d3bb3caa58f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:31.949677 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e932f423-56fc-4b9b-b2c4-8d3bb3caa58f admin admin] PATH_INFO: `/v3/users/a820200be2114b1db5d350ead6625bf1` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:31.957604 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e932f423-56fc-4b9b-b2c4-8d3bb3caa58f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a820200be2114b1db5d350ead6625bf1)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:31.960459 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e932f423-56fc-4b9b-b2c4-8d3bb3caa58f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:31.965108 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 680/1360] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:33:31 2022] POST /identity/v3/users => generated 289 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:33:31.968119 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e9440e-0a08-43b5-b875-223e6a0a0aac admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:32.005784 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e9440e-0a08-43b5-b875-223e6a0a0aac admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:32.024181 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65f420a-c42c-48e8-9c6b-f9b6baca1201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:32.026194 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f420a-c42c-48e8-9c6b-f9b6baca1201 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:32.026755 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f420a-c42c-48e8-9c6b-f9b6baca1201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:32.027664 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65f420a-c42c-48e8-9c6b-f9b6baca1201 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:32.030524 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65f420a-c42c-48e8-9c6b-f9b6baca1201 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:32.036854 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65f420a-c42c-48e8-9c6b-f9b6baca1201 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:32.065094 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:32.084938 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 681/1361] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:33:31 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:33:32.089227 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65f420a-c42c-48e8-9c6b-f9b6baca1201 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:32.114471 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65f420a-c42c-48e8-9c6b-f9b6baca1201 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:32.117544 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e932f423-56fc-4b9b-b2c4-8d3bb3caa58f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a820200be2114b1db5d350ead6625bf1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:32.128197 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292954a1-7716-4570-93a8-43e047c9c71e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:32.131878 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292954a1-7716-4570-93a8-43e047c9c71e admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:32.133404 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292954a1-7716-4570-93a8-43e047c9c71e admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:32.135033 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e932f423-56fc-4b9b-b2c4-8d3bb3caa58f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a820200be2114b1db5d350ead6625bf1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:32.136196 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e932f423-56fc-4b9b-b2c4-8d3bb3caa58f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a820200be2114b1db5d350ead6625bf1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:32.136422 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292954a1-7716-4570-93a8-43e047c9c71e admin admin] PATH_INFO: `/v3/projects/f2b2a25d39154c75b0a0589327aba5d0/users/0fdff282e1f34e0e9267ad7164b351b5/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:32.183153 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292954a1-7716-4570-93a8-43e047c9c71e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2b2a25d39154c75b0a0589327aba5d0, user_id=0fdff282e1f34e0e9267ad7164b351b5, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:32.189821 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292954a1-7716-4570-93a8-43e047c9c71e admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:32.205156 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e932f423-56fc-4b9b-b2c4-8d3bb3caa58f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a820200be2114b1db5d350ead6625bf1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:32.214943 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e932f423-56fc-4b9b-b2c4-8d3bb3caa58f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a820200be2114b1db5d350ead6625bf1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:32.218390 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 682/1362] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:33:32 2022] PUT /identity/v3/projects/f2b2a25d39154c75b0a0589327aba5d0/users/0fdff282e1f34e0e9267ad7164b351b5/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:32.224520 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292954a1-7716-4570-93a8-43e047c9c71e admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:32.228059 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-292954a1-7716-4570-93a8-43e047c9c71e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:32.241701 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 681/1363] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 03:33:31 2022] DELETE /identity/v3/users/a820200be2114b1db5d350ead6625bf1 => generated 0 bytes in 635 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:32.248901 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:32.250337 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533547fb-bac1-4696-b840-13c25fa6728c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:32.250676 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533547fb-bac1-4696-b840-13c25fa6728c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:32.250963 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533547fb-bac1-4696-b840-13c25fa6728c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:32.304346 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-533547fb-bac1-4696-b840-13c25fa6728c None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:32.305748 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-533547fb-bac1-4696-b840-13c25fa6728c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:32.373818 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292954a1-7716-4570-93a8-43e047c9c71e admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:32.479545 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 682/1364] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:32.591258 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acf7a42-8da0-419f-bd27-f03e9ac7ceac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:32.591911 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acf7a42-8da0-419f-bd27-f03e9ac7ceac admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:32.595541 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acf7a42-8da0-419f-bd27-f03e9ac7ceac admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:32.595541 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acf7a42-8da0-419f-bd27-f03e9ac7ceac admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:32.595541 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acf7a42-8da0-419f-bd27-f03e9ac7ceac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:32.598701 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acf7a42-8da0-419f-bd27-f03e9ac7ceac admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:32.600768 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533547fb-bac1-4696-b840-13c25fa6728c None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:32.605439 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-533547fb-bac1-4696-b840-13c25fa6728c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:32.624277 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 683/1365] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:33:32 2022] GET /identity/v3/roles => generated 1420 bytes in 401 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:33:32.625142 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acf7a42-8da0-419f-bd27-f03e9ac7ceac admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:32.652983 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acf7a42-8da0-419f-bd27-f03e9ac7ceac admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:32.668985 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02eca5e6-c7b3-4996-8c97-c3a92f5cda59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:32.669731 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eca5e6-c7b3-4996-8c97-c3a92f5cda59 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:32.670389 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eca5e6-c7b3-4996-8c97-c3a92f5cda59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:32.670669 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eca5e6-c7b3-4996-8c97-c3a92f5cda59 admin admin] PATH_INFO: `/v3/projects/f2b2a25d39154c75b0a0589327aba5d0/users/0fdff282e1f34e0e9267ad7164b351b5/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:32.675466 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eca5e6-c7b3-4996-8c97-c3a92f5cda59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2b2a25d39154c75b0a0589327aba5d0, user_id=0fdff282e1f34e0e9267ad7164b351b5, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:32.678153 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02eca5e6-c7b3-4996-8c97-c3a92f5cda59 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:32.708652 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 684/1366] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:33:32 2022] PUT /identity/v3/projects/f2b2a25d39154c75b0a0589327aba5d0/users/0fdff282e1f34e0e9267ad7164b351b5/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:32.720064 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42afd6b-1ad2-423e-8064-2cf7b5a30dd4 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:32.720349 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42afd6b-1ad2-423e-8064-2cf7b5a30dd4 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:32.720630 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42afd6b-1ad2-423e-8064-2cf7b5a30dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:32.810879 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-b42afd6b-1ad2-423e-8064-2cf7b5a30dd4 None None] MFA Rules not processed for user `0fdff282e1f34e0e9267ad7164b351b5`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:32.812203 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-b42afd6b-1ad2-423e-8064-2cf7b5a30dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:32.876477 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533547fb-bac1-4696-b840-13c25fa6728c None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:33.067315 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e372d409-1758-4cc0-a853-caeebc7adeec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:33.068211 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372d409-1758-4cc0-a853-caeebc7adeec service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:33.069571 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 685/1367] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:32 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 03:33:33.072069 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372d409-1758-4cc0-a853-caeebc7adeec service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:33.072537 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e372d409-1758-4cc0-a853-caeebc7adeec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:33.083930 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42afd6b-1ad2-423e-8064-2cf7b5a30dd4 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:33.085987 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-b42afd6b-1ad2-423e-8064-2cf7b5a30dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:33.118160 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:33.136231 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:33.241677 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42afd6b-1ad2-423e-8064-2cf7b5a30dd4 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:33.252595 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dde71b-0139-4635-9419-56c74f43779b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:33.253260 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dde71b-0139-4635-9419-56c74f43779b service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:33.253548 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dde71b-0139-4635-9419-56c74f43779b service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:33.253820 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dde71b-0139-4635-9419-56c74f43779b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:33.290072 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e372d409-1758-4cc0-a853-caeebc7adeec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:33.293115 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e372d409-1758-4cc0-a853-caeebc7adeec service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:33.296896 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dde71b-0139-4635-9419-56c74f43779b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:33.298414 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dde71b-0139-4635-9419-56c74f43779b service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:33.329907 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 686/1368] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:33:33 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:33:33.340886 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0adc0-4f07-44f9-9d82-1eefda0ec0b2 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:33.341183 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0adc0-4f07-44f9-9d82-1eefda0ec0b2 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:33.341626 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde0adc0-4f07-44f9-9d82-1eefda0ec0b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:33.346709 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 683/1369] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:33:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 749 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:33:33.413307 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-bde0adc0-4f07-44f9-9d82-1eefda0ec0b2 None None] MFA Rules not processed for user `f16c6a2cd8594037ad47475ed1b3161c`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:33.414590 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-bde0adc0-4f07-44f9-9d82-1eefda0ec0b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:33.499456 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e372d409-1758-4cc0-a853-caeebc7adeec service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:33.511186 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e372d409-1758-4cc0-a853-caeebc7adeec service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:33.515844 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5240fd-0061-4aff-8f2e-4e7a368b7261 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:33.517290 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5240fd-0061-4aff-8f2e-4e7a368b7261 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:33.517775 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5240fd-0061-4aff-8f2e-4e7a368b7261 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:33.518409 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5240fd-0061-4aff-8f2e-4e7a368b7261 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:33.520711 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ac5240fd-0061-4aff-8f2e-4e7a368b7261 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:33.613942 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 687/1370] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:33 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:33.632308 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde0adc0-4f07-44f9-9d82-1eefda0ec0b2 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:33.634058 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-bde0adc0-4f07-44f9-9d82-1eefda0ec0b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:33.687238 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5240fd-0061-4aff-8f2e-4e7a368b7261 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:33.688784 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5240fd-0061-4aff-8f2e-4e7a368b7261 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:33.718183 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 684/1371] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:33:33 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 03:33:33.837404 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde0adc0-4f07-44f9-9d82-1eefda0ec0b2 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:33.856882 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00602dff-6a31-4bb5-857b-05cea1ffa080 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:33.857734 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00602dff-6a31-4bb5-857b-05cea1ffa080 service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:33.858124 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00602dff-6a31-4bb5-857b-05cea1ffa080 service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:33.858505 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00602dff-6a31-4bb5-857b-05cea1ffa080 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:33.876179 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00602dff-6a31-4bb5-857b-05cea1ffa080 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:33.883864 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00602dff-6a31-4bb5-857b-05cea1ffa080 service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:33.906636 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 688/1372] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:33:33 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:33:34.006131 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5240fd-0061-4aff-8f2e-4e7a368b7261 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:34.007589 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ac5240fd-0061-4aff-8f2e-4e7a368b7261 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:34.048243 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5240fd-0061-4aff-8f2e-4e7a368b7261 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:34.065590 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ca17f2-fb22-482e-99c2-5621d1242ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:34.066601 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca17f2-fb22-482e-99c2-5621d1242ddf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:34.067812 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca17f2-fb22-482e-99c2-5621d1242ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:34.068524 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca17f2-fb22-482e-99c2-5621d1242ddf admin admin] PATH_INFO: `/v3/projects/d7961886d30c449aad4197024105a951` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:34.074207 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ca17f2-fb22-482e-99c2-5621d1242ddf admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7961886d30c449aad4197024105a951)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:34.080282 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ca17f2-fb22-482e-99c2-5621d1242ddf admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:34.189105 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d9ca17f2-fb22-482e-99c2-5621d1242ddf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7961886d30c449aad4197024105a951', 'request_id': 'req-d9ca17f2-fb22-482e-99c2-5621d1242ddf'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:34.198582 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d9ca17f2-fb22-482e-99c2-5621d1242ddf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7961886d30c449aad4197024105a951', 'request_id': 'req-d9ca17f2-fb22-482e-99c2-5621d1242ddf'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:34.217586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d9ca17f2-fb22-482e-99c2-5621d1242ddf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7961886d30c449aad4197024105a951', 'request_id': 'req-d9ca17f2-fb22-482e-99c2-5621d1242ddf'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:34.239938 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d9ca17f2-fb22-482e-99c2-5621d1242ddf admin admin] The token cache is being invalidate because project d7961886d30c449aad4197024105a951 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 03:33:34.240485 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d9ca17f2-fb22-482e-99c2-5621d1242ddf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:34.241794 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d9ca17f2-fb22-482e-99c2-5621d1242ddf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:34.246737 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 685/1373] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 03:33:34 2022] DELETE /identity/v3/projects/d7961886d30c449aad4197024105a951 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:34.252649 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88ed46d-53fe-4e56-8672-94e3b1a819d8 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:34.253397 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88ed46d-53fe-4e56-8672-94e3b1a819d8 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:34.253996 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88ed46d-53fe-4e56-8672-94e3b1a819d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:34.295541 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-e88ed46d-53fe-4e56-8672-94e3b1a819d8 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:34.296630 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e88ed46d-53fe-4e56-8672-94e3b1a819d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:34.380198 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:34.495054 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 689/1374] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:34.509022 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855bb4b7-f055-483e-a52c-f4142d607bfa None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:34.509515 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855bb4b7-f055-483e-a52c-f4142d607bfa None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:34.509900 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855bb4b7-f055-483e-a52c-f4142d607bfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:34.556493 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-855bb4b7-f055-483e-a52c-f4142d607bfa None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:34.559935 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-855bb4b7-f055-483e-a52c-f4142d607bfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:34.570571 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88ed46d-53fe-4e56-8672-94e3b1a819d8 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:34.572062 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e88ed46d-53fe-4e56-8672-94e3b1a819d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:34.603036 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 686/1375] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:34.608792 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88ed46d-53fe-4e56-8672-94e3b1a819d8 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:34.609658 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855bb4b7-f055-483e-a52c-f4142d607bfa None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:34.629461 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a38a330-46b7-499a-b39c-a6d16691851f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:34.630125 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a38a330-46b7-499a-b39c-a6d16691851f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:34.634325 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a38a330-46b7-499a-b39c-a6d16691851f admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:34.635425 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a38a330-46b7-499a-b39c-a6d16691851f admin admin] PATH_INFO: `/v3/users/082f61c1659441179e1e1b22f8e75966` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:34.642010 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a38a330-46b7-499a-b39c-a6d16691851f admin admin] RBAC: Authorizing `identity:delete_user(user_id=082f61c1659441179e1e1b22f8e75966)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:34.645136 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855bb4b7-f055-483e-a52c-f4142d607bfa None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:34.648043 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a38a330-46b7-499a-b39c-a6d16691851f admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:34.664787 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2552925b-d25d-41ae-94bd-c5f4ff1ca6b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:34.666025 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552925b-d25d-41ae-94bd-c5f4ff1ca6b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:34.666025 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552925b-d25d-41ae-94bd-c5f4ff1ca6b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:34.666223 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2552925b-d25d-41ae-94bd-c5f4ff1ca6b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:34.666900 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2552925b-d25d-41ae-94bd-c5f4ff1ca6b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:33:34.673896 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2552925b-d25d-41ae-94bd-c5f4ff1ca6b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:34.688440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2552925b-d25d-41ae-94bd-c5f4ff1ca6b5 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:34.728441 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 687/1376] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 03:33:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:33:34.729644 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2552925b-d25d-41ae-94bd-c5f4ff1ca6b5 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:34.754133 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2552925b-d25d-41ae-94bd-c5f4ff1ca6b5 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:34.757592 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1a38a330-46b7-499a-b39c-a6d16691851f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '082f61c1659441179e1e1b22f8e75966'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:34.758544 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1a38a330-46b7-499a-b39c-a6d16691851f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '082f61c1659441179e1e1b22f8e75966'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:34.759642 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1a38a330-46b7-499a-b39c-a6d16691851f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '082f61c1659441179e1e1b22f8e75966'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:34.760553 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff390060-2a4e-4280-991e-ec5b2a89846d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:34.761372 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff390060-2a4e-4280-991e-ec5b2a89846d admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:34.761816 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff390060-2a4e-4280-991e-ec5b2a89846d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:34.762288 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff390060-2a4e-4280-991e-ec5b2a89846d admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:34.763128 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff390060-2a4e-4280-991e-ec5b2a89846d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:34.765484 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff390060-2a4e-4280-991e-ec5b2a89846d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:34.782897 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:34.798940 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1a38a330-46b7-499a-b39c-a6d16691851f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '082f61c1659441179e1e1b22f8e75966'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:34.821484 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 688/1377] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:33:34 2022] POST /identity/v3/projects => generated 375 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:33:34.821940 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1a38a330-46b7-499a-b39c-a6d16691851f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '082f61c1659441179e1e1b22f8e75966'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:34.825839 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff390060-2a4e-4280-991e-ec5b2a89846d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:34.831864 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ff390060-2a4e-4280-991e-ec5b2a89846d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:34.859370 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 690/1378] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 03:33:34 2022] DELETE /identity/v3/users/082f61c1659441179e1e1b22f8e75966 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:34.866156 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b1a0f-7a3e-434f-8f09-8e64c3a2d24c None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:34.866940 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b1a0f-7a3e-434f-8f09-8e64c3a2d24c None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:34.867457 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b1a0f-7a3e-434f-8f09-8e64c3a2d24c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:34.943452 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-280b1a0f-7a3e-434f-8f09-8e64c3a2d24c None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:34.951220 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-280b1a0f-7a3e-434f-8f09-8e64c3a2d24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:35.002758 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff390060-2a4e-4280-991e-ec5b2a89846d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:35.097165 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 691/1379] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:35.113267 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a799173-3594-4b8a-bba7-54c99fb50aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:35.113901 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a799173-3594-4b8a-bba7-54c99fb50aa0 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:35.114183 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a799173-3594-4b8a-bba7-54c99fb50aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:35.114454 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a799173-3594-4b8a-bba7-54c99fb50aa0 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:35.115095 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a799173-3594-4b8a-bba7-54c99fb50aa0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:35.117296 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a799173-3594-4b8a-bba7-54c99fb50aa0 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:35.138702 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b1a0f-7a3e-434f-8f09-8e64c3a2d24c None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:35.140867 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-280b1a0f-7a3e-434f-8f09-8e64c3a2d24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:35.165050 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 689/1380] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:33:34 2022] POST /identity/v3/users => generated 288 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:33:35.166081 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a799173-3594-4b8a-bba7-54c99fb50aa0 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:35.186766 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a799173-3594-4b8a-bba7-54c99fb50aa0 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:35.205959 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee4f37-f57f-4ad8-8598-71fae7703364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:35.209466 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee4f37-f57f-4ad8-8598-71fae7703364 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:35.209697 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee4f37-f57f-4ad8-8598-71fae7703364 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:35.210036 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee4f37-f57f-4ad8-8598-71fae7703364 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:35.212600 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ee4f37-f57f-4ad8-8598-71fae7703364 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:35.218043 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ee4f37-f57f-4ad8-8598-71fae7703364 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:35.252977 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 690/1381] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:33:35 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:33:35.260553 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee4f37-f57f-4ad8-8598-71fae7703364 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:35.287071 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee4f37-f57f-4ad8-8598-71fae7703364 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:35.306977 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0ffb03-1872-4087-8547-b4cd1c1a3842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:35.307779 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ffb03-1872-4087-8547-b4cd1c1a3842 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:35.308090 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ffb03-1872-4087-8547-b4cd1c1a3842 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:35.311054 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0ffb03-1872-4087-8547-b4cd1c1a3842 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:35.311860 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0ffb03-1872-4087-8547-b4cd1c1a3842 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:35.316642 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280b1a0f-7a3e-434f-8f09-8e64c3a2d24c None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:35.317288 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0ffb03-1872-4087-8547-b4cd1c1a3842 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:35.325110 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9876717a-4b3d-4697-bda3-6d73cfdc6a75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:35.328087 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9876717a-4b3d-4697-bda3-6d73cfdc6a75 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:35.328240 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9876717a-4b3d-4697-bda3-6d73cfdc6a75 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:35.328520 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9876717a-4b3d-4697-bda3-6d73cfdc6a75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:35.344670 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:35.350245 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 691/1382] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:33:35 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:33:35.367328 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0ffb03-1872-4087-8547-b4cd1c1a3842 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:35.371652 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9876717a-4b3d-4697-bda3-6d73cfdc6a75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:35.380101 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9876717a-4b3d-4697-bda3-6d73cfdc6a75 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:35.390214 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0ffb03-1872-4087-8547-b4cd1c1a3842 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:35.402525 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 692/1383] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:33:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 03:33:35.405487 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c49282-6ad1-452c-a34c-2670f2c9f433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:35.407898 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c49282-6ad1-452c-a34c-2670f2c9f433 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:35.408446 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c49282-6ad1-452c-a34c-2670f2c9f433 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:35.409237 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c49282-6ad1-452c-a34c-2670f2c9f433 admin admin] PATH_INFO: `/v3/projects/95dfa33ba0fd4fd889a8ab63b6f573da/users/263f27b53fcd4549bfc07c72c3d63ee6/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:35.431111 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c49282-6ad1-452c-a34c-2670f2c9f433 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95dfa33ba0fd4fd889a8ab63b6f573da, user_id=263f27b53fcd4549bfc07c72c3d63ee6, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:35.434401 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c49282-6ad1-452c-a34c-2670f2c9f433 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:35.457773 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 692/1384] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:33:35 2022] PUT /identity/v3/projects/95dfa33ba0fd4fd889a8ab63b6f573da/users/263f27b53fcd4549bfc07c72c3d63ee6/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:35.470897 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ff7a2-8168-4c58-b1d0-6c6e55f0df4e None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:35.471449 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ff7a2-8168-4c58-b1d0-6c6e55f0df4e None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:35.471911 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56ff7a2-8168-4c58-b1d0-6c6e55f0df4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:35.538481 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-f56ff7a2-8168-4c58-b1d0-6c6e55f0df4e None None] MFA Rules not processed for user `263f27b53fcd4549bfc07c72c3d63ee6`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:35.539913 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f56ff7a2-8168-4c58-b1d0-6c6e55f0df4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:35.726679 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 693/1385] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:35 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:35.730885 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c49282-6ad1-452c-a34c-2670f2c9f433 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:35.837887 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:35.908351 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c49282-6ad1-452c-a34c-2670f2c9f433 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:35.922779 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dc16d0-f77b-4ca5-8226-66af63b481e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:35.923978 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc16d0-f77b-4ca5-8226-66af63b481e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:35.924446 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc16d0-f77b-4ca5-8226-66af63b481e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:35.924728 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dc16d0-f77b-4ca5-8226-66af63b481e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:35.926552 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dc16d0-f77b-4ca5-8226-66af63b481e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:35.930670 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dc16d0-f77b-4ca5-8226-66af63b481e0 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:35.993280 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 693/1386] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:33:35 2022] POST /identity/v3/projects => generated 377 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:33:35.996993 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56ff7a2-8168-4c58-b1d0-6c6e55f0df4e None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:36.015201 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56ff7a2-8168-4c58-b1d0-6c6e55f0df4e None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:36.147385 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541b58d7-beb7-45e4-b1e3-2bd23d4a26bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:36.148043 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b58d7-beb7-45e4-b1e3-2bd23d4a26bb admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:36.148325 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b58d7-beb7-45e4-b1e3-2bd23d4a26bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:36.148603 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541b58d7-beb7-45e4-b1e3-2bd23d4a26bb admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:36.149262 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541b58d7-beb7-45e4-b1e3-2bd23d4a26bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:36.152819 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541b58d7-beb7-45e4-b1e3-2bd23d4a26bb admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:36.204737 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 694/1387] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:33:35 2022] POST /identity/v3/users => generated 295 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:33:36.208492 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dc16d0-f77b-4ca5-8226-66af63b481e0 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:36.223553 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dc16d0-f77b-4ca5-8226-66af63b481e0 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:36.235239 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37e4d1f-abf1-49e9-b064-8fe3cf8dfc4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:36.235239 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37e4d1f-abf1-49e9-b064-8fe3cf8dfc4b admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:36.236270 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37e4d1f-abf1-49e9-b064-8fe3cf8dfc4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:36.236577 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37e4d1f-abf1-49e9-b064-8fe3cf8dfc4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:36.237871 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37e4d1f-abf1-49e9-b064-8fe3cf8dfc4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:36.242493 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37e4d1f-abf1-49e9-b064-8fe3cf8dfc4b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:36.268447 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 694/1388] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:33:36 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:33:36.272565 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541b58d7-beb7-45e4-b1e3-2bd23d4a26bb admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:36.286376 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541b58d7-beb7-45e4-b1e3-2bd23d4a26bb admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:36.297454 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62b3502-3eb9-4e60-b557-6d70a695f520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:36.298386 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b3502-3eb9-4e60-b557-6d70a695f520 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:36.298670 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b3502-3eb9-4e60-b557-6d70a695f520 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:36.298947 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62b3502-3eb9-4e60-b557-6d70a695f520 admin admin] PATH_INFO: `/v3/projects/7a547cfa2eaa444b9a7234fc2555779e/users/21c2ee4d66ee4cbdacc52596ebcf95d9/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:36.336014 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62b3502-3eb9-4e60-b557-6d70a695f520 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a547cfa2eaa444b9a7234fc2555779e, user_id=21c2ee4d66ee4cbdacc52596ebcf95d9, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:36.349580 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62b3502-3eb9-4e60-b557-6d70a695f520 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:36.378511 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 695/1389] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:33:36 2022] PUT /identity/v3/projects/7a547cfa2eaa444b9a7234fc2555779e/users/21c2ee4d66ee4cbdacc52596ebcf95d9/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:36.382864 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37e4d1f-abf1-49e9-b064-8fe3cf8dfc4b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:36.412328 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37e4d1f-abf1-49e9-b064-8fe3cf8dfc4b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:36.446747 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62b3502-3eb9-4e60-b557-6d70a695f520 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:36.449445 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-c62b3502-3eb9-4e60-b557-6d70a695f520 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:36.560433 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84e622c-837a-4836-9019-452cb7a908f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:36.561241 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e622c-837a-4836-9019-452cb7a908f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:36.561632 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e622c-837a-4836-9019-452cb7a908f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:36.562352 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e622c-837a-4836-9019-452cb7a908f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:36.562824 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84e622c-837a-4836-9019-452cb7a908f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:36.564676 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84e622c-837a-4836-9019-452cb7a908f8 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:36.588311 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 695/1390] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:33:36 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:33:36.589533 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84e622c-837a-4836-9019-452cb7a908f8 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:36.605622 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84e622c-837a-4836-9019-452cb7a908f8 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:36.617970 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a5ed5b-b5b6-47eb-995c-6cb65c1344fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:36.618770 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a5ed5b-b5b6-47eb-995c-6cb65c1344fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:36.619159 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a5ed5b-b5b6-47eb-995c-6cb65c1344fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:36.619662 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a5ed5b-b5b6-47eb-995c-6cb65c1344fa admin admin] PATH_INFO: `/v3/projects/7a547cfa2eaa444b9a7234fc2555779e/users/21c2ee4d66ee4cbdacc52596ebcf95d9/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:36.625889 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a5ed5b-b5b6-47eb-995c-6cb65c1344fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a547cfa2eaa444b9a7234fc2555779e, user_id=21c2ee4d66ee4cbdacc52596ebcf95d9, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:36.628866 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:36.640699 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a5ed5b-b5b6-47eb-995c-6cb65c1344fa admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:36.664101 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 696/1391] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:33:36 2022] PUT /identity/v3/projects/7a547cfa2eaa444b9a7234fc2555779e/users/21c2ee4d66ee4cbdacc52596ebcf95d9/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:36.664307 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62b3502-3eb9-4e60-b557-6d70a695f520 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:36.665898 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3b8481-dc62-46f1-af61-f623b0df0444 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:36.666326 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3b8481-dc62-46f1-af61-f623b0df0444 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:36.666706 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3b8481-dc62-46f1-af61-f623b0df0444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:36.724805 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-cd3b8481-dc62-46f1-af61-f623b0df0444 None None] MFA Rules not processed for user `21c2ee4d66ee4cbdacc52596ebcf95d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:36.726218 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-cd3b8481-dc62-46f1-af61-f623b0df0444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:36.831021 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f204905-be9f-4183-97cf-c1b625df9caa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:36.832597 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f204905-be9f-4183-97cf-c1b625df9caa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:36.833284 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f204905-be9f-4183-97cf-c1b625df9caa admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:36.833862 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f204905-be9f-4183-97cf-c1b625df9caa admin admin] PATH_INFO: `/v3/projects/73435b08d77c4f65a36325526a2baf29` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:36.835434 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f204905-be9f-4183-97cf-c1b625df9caa admin admin] RBAC: Authorizing `identity:delete_project(project_id=73435b08d77c4f65a36325526a2baf29)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:36.839023 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f204905-be9f-4183-97cf-c1b625df9caa admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:36.883276 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 697/1392] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:36 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:36.896019 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3b8481-dc62-46f1-af61-f623b0df0444 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:36.897855 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-cd3b8481-dc62-46f1-af61-f623b0df0444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:36.951519 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:36.957499 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-0f204905-be9f-4183-97cf-c1b625df9caa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73435b08d77c4f65a36325526a2baf29', 'request_id': 'req-0f204905-be9f-4183-97cf-c1b625df9caa'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:36.970308 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-0f204905-be9f-4183-97cf-c1b625df9caa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73435b08d77c4f65a36325526a2baf29', 'request_id': 'req-0f204905-be9f-4183-97cf-c1b625df9caa'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:36.979802 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-0f204905-be9f-4183-97cf-c1b625df9caa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73435b08d77c4f65a36325526a2baf29', 'request_id': 'req-0f204905-be9f-4183-97cf-c1b625df9caa'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:37.011752 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-0f204905-be9f-4183-97cf-c1b625df9caa admin admin] The token cache is being invalidate because project 73435b08d77c4f65a36325526a2baf29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 03:33:37.013604 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3b8481-dc62-46f1-af61-f623b0df0444 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:37.016463 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-0f204905-be9f-4183-97cf-c1b625df9caa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:37.018063 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-0f204905-be9f-4183-97cf-c1b625df9caa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:37.026745 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 696/1393] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 03:33:36 2022] DELETE /identity/v3/projects/73435b08d77c4f65a36325526a2baf29 => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 03:33:37.030469 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f204905-be9f-4183-97cf-c1b625df9caa admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:37.035076 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-0f204905-be9f-4183-97cf-c1b625df9caa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:37.129024 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9a8426-0354-40c0-b858-25ef2ddb75cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:37.129786 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a8426-0354-40c0-b858-25ef2ddb75cd service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:37.130064 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a8426-0354-40c0-b858-25ef2ddb75cd service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:37.130334 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9a8426-0354-40c0-b858-25ef2ddb75cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:37.137680 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3b9a8426-0354-40c0-b858-25ef2ddb75cd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:37.218111 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f204905-be9f-4183-97cf-c1b625df9caa admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:37.231557 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e166fc70-d07b-44c1-83bc-8c7f8ae5f68e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:37.236660 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e166fc70-d07b-44c1-83bc-8c7f8ae5f68e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:37.236954 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e166fc70-d07b-44c1-83bc-8c7f8ae5f68e admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:37.237242 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e166fc70-d07b-44c1-83bc-8c7f8ae5f68e admin admin] PATH_INFO: `/v3/users/894a90c7c5ac4e5f98415f2a12645e69` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:37.238569 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e166fc70-d07b-44c1-83bc-8c7f8ae5f68e admin admin] RBAC: Authorizing `identity:delete_user(user_id=894a90c7c5ac4e5f98415f2a12645e69)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:37.242982 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e166fc70-d07b-44c1-83bc-8c7f8ae5f68e admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:37.290873 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9a8426-0354-40c0-b858-25ef2ddb75cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:37.297773 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b9a8426-0354-40c0-b858-25ef2ddb75cd service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:37.316002 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:37.328624 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 698/1394] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:33:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:33:37.363627 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e166fc70-d07b-44c1-83bc-8c7f8ae5f68e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '894a90c7c5ac4e5f98415f2a12645e69'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:37.364640 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e166fc70-d07b-44c1-83bc-8c7f8ae5f68e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '894a90c7c5ac4e5f98415f2a12645e69'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:37.365518 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e166fc70-d07b-44c1-83bc-8c7f8ae5f68e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '894a90c7c5ac4e5f98415f2a12645e69'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:37.406247 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e166fc70-d07b-44c1-83bc-8c7f8ae5f68e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '894a90c7c5ac4e5f98415f2a12645e69'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:37.420254 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e166fc70-d07b-44c1-83bc-8c7f8ae5f68e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '894a90c7c5ac4e5f98415f2a12645e69'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:37.441614 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 697/1395] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 03:33:37 2022] DELETE /identity/v3/users/894a90c7c5ac4e5f98415f2a12645e69 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:38.672161 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9a8426-0354-40c0-b858-25ef2ddb75cd service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:38.692942 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9a8426-0354-40c0-b858-25ef2ddb75cd service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:38.823446 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d0bf2b-7421-47c4-978a-947e950b6619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:38.824529 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d0bf2b-7421-47c4-978a-947e950b6619 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:38.824947 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d0bf2b-7421-47c4-978a-947e950b6619 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:38.825474 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d0bf2b-7421-47c4-978a-947e950b6619 admin admin] PATH_INFO: `/v3/projects/158f6689158d475c96e7aa783468accc` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:38.826926 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d0bf2b-7421-47c4-978a-947e950b6619 admin admin] RBAC: Authorizing `identity:delete_project(project_id=158f6689158d475c96e7aa783468accc)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:38.829864 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d0bf2b-7421-47c4-978a-947e950b6619 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:38.908313 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:38.964545 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-41d0bf2b-7421-47c4-978a-947e950b6619 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '158f6689158d475c96e7aa783468accc', 'request_id': 'req-41d0bf2b-7421-47c4-978a-947e950b6619'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:38.987371 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-41d0bf2b-7421-47c4-978a-947e950b6619 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '158f6689158d475c96e7aa783468accc', 'request_id': 'req-41d0bf2b-7421-47c4-978a-947e950b6619'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:38.999366 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-41d0bf2b-7421-47c4-978a-947e950b6619 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '158f6689158d475c96e7aa783468accc', 'request_id': 'req-41d0bf2b-7421-47c4-978a-947e950b6619'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:39.024615 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-41d0bf2b-7421-47c4-978a-947e950b6619 admin admin] The token cache is being invalidate because project 158f6689158d475c96e7aa783468accc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 03:33:39.024974 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-41d0bf2b-7421-47c4-978a-947e950b6619 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:39.026217 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-41d0bf2b-7421-47c4-978a-947e950b6619 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:39.033666 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4f1c5-7889-4ffa-9e67-114244166faa None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:39.034154 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4f1c5-7889-4ffa-9e67-114244166faa None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:39.034711 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4f1c5-7889-4ffa-9e67-114244166faa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:39.043998 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 699/1396] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 03:33:38 2022] DELETE /identity/v3/projects/158f6689158d475c96e7aa783468accc => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:39.067719 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-86c4f1c5-7889-4ffa-9e67-114244166faa None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:39.069071 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-86c4f1c5-7889-4ffa-9e67-114244166faa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:39.189139 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 698/1397] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:39.193350 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb8edf-b57c-4ef5-aa76-0418f9ecacf3 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:39.193866 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb8edf-b57c-4ef5-aa76-0418f9ecacf3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:39.194311 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcb8edf-b57c-4ef5-aa76-0418f9ecacf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:39.231698 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-3dcb8edf-b57c-4ef5-aa76-0418f9ecacf3 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:39.235376 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3dcb8edf-b57c-4ef5-aa76-0418f9ecacf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:39.248665 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 700/1398] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:39.249558 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c4f1c5-7889-4ffa-9e67-114244166faa None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:39.264146 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c4f1c5-7889-4ffa-9e67-114244166faa None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:39.271471 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd4c1b2-3d1f-49fc-88c5-63084f9a45ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:39.272608 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4c1b2-3d1f-49fc-88c5-63084f9a45ed admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:39.272608 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4c1b2-3d1f-49fc-88c5-63084f9a45ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:39.272761 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd4c1b2-3d1f-49fc-88c5-63084f9a45ed admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:39.273385 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd4c1b2-3d1f-49fc-88c5-63084f9a45ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:33:39.273747 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd4c1b2-3d1f-49fc-88c5-63084f9a45ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:39.275764 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd4c1b2-3d1f-49fc-88c5-63084f9a45ed admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:39.295992 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 699/1399] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 03:33:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:33:39.298124 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcb8edf-b57c-4ef5-aa76-0418f9ecacf3 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:39.308016 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcb8edf-b57c-4ef5-aa76-0418f9ecacf3 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:39.314480 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26e98d4-6d6c-4ee4-91ba-eddd957614c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:39.315253 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26e98d4-6d6c-4ee4-91ba-eddd957614c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:39.315633 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26e98d4-6d6c-4ee4-91ba-eddd957614c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:39.316042 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26e98d4-6d6c-4ee4-91ba-eddd957614c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:39.317340 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26e98d4-6d6c-4ee4-91ba-eddd957614c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:39.320251 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26e98d4-6d6c-4ee4-91ba-eddd957614c3 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:39.381694 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd4c1b2-3d1f-49fc-88c5-63084f9a45ed admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:39.395147 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 701/1400] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:33:39 2022] POST /identity/v3/projects => generated 387 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:33:39.401072 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd4c1b2-3d1f-49fc-88c5-63084f9a45ed admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:39.592790 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e300b7-cee3-4028-ac33-04ccb4433a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:39.634186 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e300b7-cee3-4028-ac33-04ccb4433a90 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:39.634530 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e300b7-cee3-4028-ac33-04ccb4433a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:39.634811 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e300b7-cee3-4028-ac33-04ccb4433a90 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:39.635569 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e300b7-cee3-4028-ac33-04ccb4433a90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:39.637811 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e300b7-cee3-4028-ac33-04ccb4433a90 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:39.715178 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 700/1401] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:33:39 2022] POST /identity/v3/users => generated 294 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:33:39.725322 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26e98d4-6d6c-4ee4-91ba-eddd957614c3 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:39.746550 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26e98d4-6d6c-4ee4-91ba-eddd957614c3 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:39.763245 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2704981-6a9b-43f5-9c3f-be985d0e0648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:39.763871 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2704981-6a9b-43f5-9c3f-be985d0e0648 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:39.764377 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2704981-6a9b-43f5-9c3f-be985d0e0648 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:39.764698 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2704981-6a9b-43f5-9c3f-be985d0e0648 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:39.765449 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2704981-6a9b-43f5-9c3f-be985d0e0648 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:39.768081 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2704981-6a9b-43f5-9c3f-be985d0e0648 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:39.806463 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 702/1402] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:33:39 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:33:39.810752 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e300b7-cee3-4028-ac33-04ccb4433a90 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:39.833433 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e300b7-cee3-4028-ac33-04ccb4433a90 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:39.854257 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e85a16-91b9-43c2-a3f4-21ff94139765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:39.854835 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e85a16-91b9-43c2-a3f4-21ff94139765 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:39.855188 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e85a16-91b9-43c2-a3f4-21ff94139765 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:39.855427 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e85a16-91b9-43c2-a3f4-21ff94139765 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:39.856044 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e85a16-91b9-43c2-a3f4-21ff94139765 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:39.858610 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e85a16-91b9-43c2-a3f4-21ff94139765 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:39.892748 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 701/1403] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:33:39 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:33:39.893923 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2704981-6a9b-43f5-9c3f-be985d0e0648 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:39.910498 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2704981-6a9b-43f5-9c3f-be985d0e0648 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:39.923589 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e43975f-5c22-4a85-aa90-699f270c42a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:39.924324 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e43975f-5c22-4a85-aa90-699f270c42a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:39.924601 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e43975f-5c22-4a85-aa90-699f270c42a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:39.924867 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e43975f-5c22-4a85-aa90-699f270c42a4 admin admin] PATH_INFO: `/v3/projects/02805fc23cc54acf9fd68a8a28c41508/users/f191a4ff90a8466dafc1dc7617df71e2/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:39.951082 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e43975f-5c22-4a85-aa90-699f270c42a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02805fc23cc54acf9fd68a8a28c41508, user_id=f191a4ff90a8466dafc1dc7617df71e2, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:39.965199 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e43975f-5c22-4a85-aa90-699f270c42a4 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:40.007683 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 703/1404] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:33:39 2022] PUT /identity/v3/projects/02805fc23cc54acf9fd68a8a28c41508/users/f191a4ff90a8466dafc1dc7617df71e2/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:40.018451 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eda633-6472-4c33-a6eb-f9a27057b052 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:40.018774 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eda633-6472-4c33-a6eb-f9a27057b052 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:40.019162 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69eda633-6472-4c33-a6eb-f9a27057b052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:40.048901 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:40.082692 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-69eda633-6472-4c33-a6eb-f9a27057b052 None None] MFA Rules not processed for user `f191a4ff90a8466dafc1dc7617df71e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:40.086198 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-69eda633-6472-4c33-a6eb-f9a27057b052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:40.101731 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ed6c18-dd57-489c-8f66-22ad19788d68 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:40.102014 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ed6c18-dd57-489c-8f66-22ad19788d68 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:40.102473 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ed6c18-dd57-489c-8f66-22ad19788d68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:40.171605 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-93ed6c18-dd57-489c-8f66-22ad19788d68 None None] MFA Rules not processed for user `7984dc1d452a4c998b44e6d26a299df8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:40.174136 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-93ed6c18-dd57-489c-8f66-22ad19788d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:40.237577 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 702/1405] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:40 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:40.241866 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eda633-6472-4c33-a6eb-f9a27057b052 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:40.316826 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 704/1406] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 03:33:40.334399 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ed6c18-dd57-489c-8f66-22ad19788d68 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:40.336081 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-93ed6c18-dd57-489c-8f66-22ad19788d68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:40.336432 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69eda633-6472-4c33-a6eb-f9a27057b052 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:40.341445 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d535221b-c696-4328-878f-d4c03779af75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:40.342058 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535221b-c696-4328-878f-d4c03779af75 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:40.342336 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535221b-c696-4328-878f-d4c03779af75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:40.342607 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d535221b-c696-4328-878f-d4c03779af75 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:40.343237 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d535221b-c696-4328-878f-d4c03779af75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:40.345912 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d535221b-c696-4328-878f-d4c03779af75 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:40.386496 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 703/1407] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:33:40 2022] POST /identity/v3/projects => generated 387 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 03:33:40.392716 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d535221b-c696-4328-878f-d4c03779af75 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:40.414055 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:40.466413 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ed6c18-dd57-489c-8f66-22ad19788d68 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:40.562019 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d535221b-c696-4328-878f-d4c03779af75 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:40.581749 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a168bf4a-a076-4105-b4a6-74143721eb04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:40.584047 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a168bf4a-a076-4105-b4a6-74143721eb04 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:40.584394 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a168bf4a-a076-4105-b4a6-74143721eb04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:40.587155 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a168bf4a-a076-4105-b4a6-74143721eb04 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:40.588011 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a168bf4a-a076-4105-b4a6-74143721eb04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:40.604920 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a168bf4a-a076-4105-b4a6-74143721eb04 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:40.635334 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40e00e0-4dac-4c38-97d0-9c0f0af62dd6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:40.636122 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40e00e0-4dac-4c38-97d0-9c0f0af62dd6 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:40.636604 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40e00e0-4dac-4c38-97d0-9c0f0af62dd6 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:40.636971 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40e00e0-4dac-4c38-97d0-9c0f0af62dd6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:40.664861 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 704/1408] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:33:40 2022] POST /identity/v3/users => generated 300 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:33:40.669399 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a168bf4a-a076-4105-b4a6-74143721eb04 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:40.681933 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a168bf4a-a076-4105-b4a6-74143721eb04 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:40.693189 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58237c46-a43e-4ed2-82c3-ea3de96c7cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:40.693954 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58237c46-a43e-4ed2-82c3-ea3de96c7cd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:40.694233 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58237c46-a43e-4ed2-82c3-ea3de96c7cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:40.694558 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58237c46-a43e-4ed2-82c3-ea3de96c7cd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:40.695272 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58237c46-a43e-4ed2-82c3-ea3de96c7cd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:40.699556 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58237c46-a43e-4ed2-82c3-ea3de96c7cd5 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:40.712912 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 705/1409] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:33:40 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 03:33:40.717940 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58237c46-a43e-4ed2-82c3-ea3de96c7cd5 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:40.729303 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:40.744939 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58237c46-a43e-4ed2-82c3-ea3de96c7cd5 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:40.749338 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40e00e0-4dac-4c38-97d0-9c0f0af62dd6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:40.750970 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40e00e0-4dac-4c38-97d0-9c0f0af62dd6 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:40.759662 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4696cd02-6703-4d49-892a-557ca0c86857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:40.766246 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4696cd02-6703-4d49-892a-557ca0c86857 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:40.768252 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4696cd02-6703-4d49-892a-557ca0c86857 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:40.768662 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4696cd02-6703-4d49-892a-557ca0c86857 admin admin] PATH_INFO: `/v3/projects/d67a3c547637490097cb05d74d62383c/users/b9a22711d03e418bb8323d0d341c2fb8/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:40.774554 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 705/1410] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:33:40 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 03:33:40.805377 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4696cd02-6703-4d49-892a-557ca0c86857 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d67a3c547637490097cb05d74d62383c, user_id=b9a22711d03e418bb8323d0d341c2fb8, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:40.811526 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4696cd02-6703-4d49-892a-557ca0c86857 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:40.850483 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 706/1411] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:33:40 2022] PUT /identity/v3/projects/d67a3c547637490097cb05d74d62383c/users/b9a22711d03e418bb8323d0d341c2fb8/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:40.854447 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40e00e0-4dac-4c38-97d0-9c0f0af62dd6 service nova] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:40.873448 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40e00e0-4dac-4c38-97d0-9c0f0af62dd6 service nova] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:40.995493 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59140bc0-ee24-4484-88ad-53107852ccd5 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:40.995784 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59140bc0-ee24-4484-88ad-53107852ccd5 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:40.996080 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59140bc0-ee24-4484-88ad-53107852ccd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:41.021158 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bee245c-f164-4250-b7b0-ce4266c7affb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:41.021802 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee245c-f164-4250-b7b0-ce4266c7affb admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:41.022079 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee245c-f164-4250-b7b0-ce4266c7affb admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:41.022346 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bee245c-f164-4250-b7b0-ce4266c7affb admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:41.022993 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bee245c-f164-4250-b7b0-ce4266c7affb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:41.024663 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bee245c-f164-4250-b7b0-ce4266c7affb admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:41.084441 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 706/1412] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:33:40 2022] GET /identity/v3/roles => generated 1420 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:33:41.089443 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bee245c-f164-4250-b7b0-ce4266c7affb admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:41.098901 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-59140bc0-ee24-4484-88ad-53107852ccd5 None None] MFA Rules not processed for user `7fdb70174b404f4abf0bc220d5dd4bb1`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:41.100187 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-59140bc0-ee24-4484-88ad-53107852ccd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:41.109868 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bee245c-f164-4250-b7b0-ce4266c7affb admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:41.125544 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41843e02-57dc-4cd4-8ab9-ae1a87960c6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:41.127075 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41843e02-57dc-4cd4-8ab9-ae1a87960c6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:41.128293 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41843e02-57dc-4cd4-8ab9-ae1a87960c6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:41.129512 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41843e02-57dc-4cd4-8ab9-ae1a87960c6d admin admin] PATH_INFO: `/v3/projects/d67a3c547637490097cb05d74d62383c/users/b9a22711d03e418bb8323d0d341c2fb8/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:41.133960 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41843e02-57dc-4cd4-8ab9-ae1a87960c6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d67a3c547637490097cb05d74d62383c, user_id=b9a22711d03e418bb8323d0d341c2fb8, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:41.144359 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41843e02-57dc-4cd4-8ab9-ae1a87960c6d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:41.168559 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 707/1413] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:33:41 2022] PUT /identity/v3/projects/d67a3c547637490097cb05d74d62383c/users/b9a22711d03e418bb8323d0d341c2fb8/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:41.171348 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41843e02-57dc-4cd4-8ab9-ae1a87960c6d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:41.175126 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-41843e02-57dc-4cd4-8ab9-ae1a87960c6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:41.207160 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:41.279930 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 707/1414] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:40 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 03:33:41.286359 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc0149-eb48-4901-b373-b7f4868b194a None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:41.286775 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc0149-eb48-4901-b373-b7f4868b194a None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:41.287280 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cc0149-eb48-4901-b373-b7f4868b194a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:41.347541 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-c0cc0149-eb48-4901-b373-b7f4868b194a None None] MFA Rules not processed for user `b9a22711d03e418bb8323d0d341c2fb8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:41.349036 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-c0cc0149-eb48-4901-b373-b7f4868b194a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:41.357975 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41843e02-57dc-4cd4-8ab9-ae1a87960c6d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:41.366027 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5c1efa-9485-40d7-bfcc-89702d8128e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:41.367042 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c1efa-9485-40d7-bfcc-89702d8128e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:41.367482 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c1efa-9485-40d7-bfcc-89702d8128e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:41.368001 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5c1efa-9485-40d7-bfcc-89702d8128e1 admin admin] PATH_INFO: `/v3/users/8d9b08ab10f64ccea3015a7570ab0b84` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:41.370077 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5c1efa-9485-40d7-bfcc-89702d8128e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d9b08ab10f64ccea3015a7570ab0b84)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:41.376958 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5c1efa-9485-40d7-bfcc-89702d8128e1 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:41.505719 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2e5c1efa-9485-40d7-bfcc-89702d8128e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d9b08ab10f64ccea3015a7570ab0b84'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:41.527926 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 708/1415] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:41 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:41.529899 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2e5c1efa-9485-40d7-bfcc-89702d8128e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d9b08ab10f64ccea3015a7570ab0b84'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:41.531409 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2e5c1efa-9485-40d7-bfcc-89702d8128e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d9b08ab10f64ccea3015a7570ab0b84'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:41.533941 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df12bb-5af4-4c7e-8558-51c526ca5250 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:41.534235 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df12bb-5af4-4c7e-8558-51c526ca5250 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:41.534687 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df12bb-5af4-4c7e-8558-51c526ca5250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:41.550824 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2e5c1efa-9485-40d7-bfcc-89702d8128e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d9b08ab10f64ccea3015a7570ab0b84'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:41.569499 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2e5c1efa-9485-40d7-bfcc-89702d8128e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d9b08ab10f64ccea3015a7570ab0b84'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:41.582254 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 708/1416] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 03:33:41 2022] DELETE /identity/v3/users/8d9b08ab10f64ccea3015a7570ab0b84 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:41.583287 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5c1efa-9485-40d7-bfcc-89702d8128e1 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:41.587793 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-a7df12bb-5af4-4c7e-8558-51c526ca5250 None None] MFA Rules not processed for user `7fdb70174b404f4abf0bc220d5dd4bb1`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:41.588958 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2e5c1efa-9485-40d7-bfcc-89702d8128e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:41.589068 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a7df12bb-5af4-4c7e-8558-51c526ca5250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:41.695916 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:41.723763 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 709/1417] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:41 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:41.725350 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7493b08f-3b26-4495-8731-433e3fdca4a2 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:41.725751 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7493b08f-3b26-4495-8731-433e3fdca4a2 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:41.726132 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7493b08f-3b26-4495-8731-433e3fdca4a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:41.745847 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5c1efa-9485-40d7-bfcc-89702d8128e1 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:41.768133 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8683fed-60d5-4ecb-b84a-b2be2146106b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:41.768973 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8683fed-60d5-4ecb-b84a-b2be2146106b service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:41.770087 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8683fed-60d5-4ecb-b84a-b2be2146106b service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:41.770830 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8683fed-60d5-4ecb-b84a-b2be2146106b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:41.775458 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f8683fed-60d5-4ecb-b84a-b2be2146106b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:41.797050 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-7493b08f-3b26-4495-8731-433e3fdca4a2 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:41.802380 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-7493b08f-3b26-4495-8731-433e3fdca4a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:41.899999 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:41.938274 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 710/1418] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:41.940494 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7493b08f-3b26-4495-8731-433e3fdca4a2 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:41.941896 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-7493b08f-3b26-4495-8731-433e3fdca4a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:41.981878 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8683fed-60d5-4ecb-b84a-b2be2146106b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:41.986135 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8683fed-60d5-4ecb-b84a-b2be2146106b service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:42.009422 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 709/1419] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:33:41 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:33:42.015895 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb31d592-900c-40d5-b415-7b0c53859b59 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:42.016203 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb31d592-900c-40d5-b415-7b0c53859b59 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:42.016491 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb31d592-900c-40d5-b415-7b0c53859b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:42.069088 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-eb31d592-900c-40d5-b415-7b0c53859b59 None None] MFA Rules not processed for user `f16c6a2cd8594037ad47475ed1b3161c`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:42.074496 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-eb31d592-900c-40d5-b415-7b0c53859b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:42.142920 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7493b08f-3b26-4495-8731-433e3fdca4a2 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:42.158626 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeee5f95-559d-4bf8-84c2-85476636f565 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:42.159265 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee5f95-559d-4bf8-84c2-85476636f565 service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:42.159547 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee5f95-559d-4bf8-84c2-85476636f565 service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:42.161907 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee5f95-559d-4bf8-84c2-85476636f565 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:42.165364 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-eeee5f95-559d-4bf8-84c2-85476636f565 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:42.203285 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeee5f95-559d-4bf8-84c2-85476636f565 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:42.206416 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeee5f95-559d-4bf8-84c2-85476636f565 service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:42.240011 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 711/1420] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:33:41 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:33:42.273298 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeee5f95-559d-4bf8-84c2-85476636f565 service nova] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:42.280944 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-eeee5f95-559d-4bf8-84c2-85476636f565 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:42.341300 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 710/1421] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:42 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:42.368650 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb31d592-900c-40d5-b415-7b0c53859b59 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:42.372579 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-eb31d592-900c-40d5-b415-7b0c53859b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:42.401929 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb31d592-900c-40d5-b415-7b0c53859b59 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:42.410554 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b77d91f-42ef-47a5-a037-b8ccd5b43b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:42.412369 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b77d91f-42ef-47a5-a037-b8ccd5b43b02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:42.412882 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b77d91f-42ef-47a5-a037-b8ccd5b43b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:42.413308 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b77d91f-42ef-47a5-a037-b8ccd5b43b02 admin admin] PATH_INFO: `/v3/users/f191a4ff90a8466dafc1dc7617df71e2` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:42.416133 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b77d91f-42ef-47a5-a037-b8ccd5b43b02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f191a4ff90a8466dafc1dc7617df71e2)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:42.418583 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b77d91f-42ef-47a5-a037-b8ccd5b43b02 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:42.520354 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeee5f95-559d-4bf8-84c2-85476636f565 service nova] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:42.539745 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf225a5-e50a-4a23-9882-3594517e3ea1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:42.540483 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf225a5-e50a-4a23-9882-3594517e3ea1 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:42.540818 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf225a5-e50a-4a23-9882-3594517e3ea1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:42.541216 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf225a5-e50a-4a23-9882-3594517e3ea1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:42.571470 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf225a5-e50a-4a23-9882-3594517e3ea1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:42.577584 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf225a5-e50a-4a23-9882-3594517e3ea1 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:42.623435 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 712/1422] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:33:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:33:42.637209 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5b77d91f-42ef-47a5-a037-b8ccd5b43b02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f191a4ff90a8466dafc1dc7617df71e2'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:42.646954 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5b77d91f-42ef-47a5-a037-b8ccd5b43b02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f191a4ff90a8466dafc1dc7617df71e2'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:42.647936 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5b77d91f-42ef-47a5-a037-b8ccd5b43b02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f191a4ff90a8466dafc1dc7617df71e2'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:42.672074 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5b77d91f-42ef-47a5-a037-b8ccd5b43b02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f191a4ff90a8466dafc1dc7617df71e2'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:42.686540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5b77d91f-42ef-47a5-a037-b8ccd5b43b02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f191a4ff90a8466dafc1dc7617df71e2'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:42.709083 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 711/1423] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 03:33:42 2022] DELETE /identity/v3/users/f191a4ff90a8466dafc1dc7617df71e2 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:42.713516 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22845d58-8178-4edb-8b04-99483479c89b None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:42.713808 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22845d58-8178-4edb-8b04-99483479c89b None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:42.714103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22845d58-8178-4edb-8b04-99483479c89b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:42.761250 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-22845d58-8178-4edb-8b04-99483479c89b None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:42.771876 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-22845d58-8178-4edb-8b04-99483479c89b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:42.829719 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:42.910552 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b77d91f-42ef-47a5-a037-b8ccd5b43b02 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:42.918928 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5b77d91f-42ef-47a5-a037-b8ccd5b43b02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:42.953546 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 713/1424] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:43.066028 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b77d91f-42ef-47a5-a037-b8ccd5b43b02 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:43.073085 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71051d6f-ae33-4468-8002-11016dae6a4b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:43.073720 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71051d6f-ae33-4468-8002-11016dae6a4b service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:43.074061 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71051d6f-ae33-4468-8002-11016dae6a4b service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:43.074336 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71051d6f-ae33-4468-8002-11016dae6a4b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:43.076163 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-71051d6f-ae33-4468-8002-11016dae6a4b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:43.083876 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:43.221682 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71051d6f-ae33-4468-8002-11016dae6a4b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:43.223619 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71051d6f-ae33-4468-8002-11016dae6a4b service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:43.240388 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 712/1425] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:33:42 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:33:43.280501 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22845d58-8178-4edb-8b04-99483479c89b None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:43.283468 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-22845d58-8178-4edb-8b04-99483479c89b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:43.431770 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22845d58-8178-4edb-8b04-99483479c89b None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:43.444288 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3754a15-17a4-4574-a58c-c0de6b5f810e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:43.444930 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3754a15-17a4-4574-a58c-c0de6b5f810e service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:43.445241 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3754a15-17a4-4574-a58c-c0de6b5f810e service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:43.445471 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3754a15-17a4-4574-a58c-c0de6b5f810e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:43.476462 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3754a15-17a4-4574-a58c-c0de6b5f810e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:43.477949 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3754a15-17a4-4574-a58c-c0de6b5f810e service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:43.498273 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 714/1426] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:33:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:33:43.737819 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef691fe5-eee9-49ea-a6ba-30d54b9685ce None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:43.738160 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef691fe5-eee9-49ea-a6ba-30d54b9685ce None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:43.738440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef691fe5-eee9-49ea-a6ba-30d54b9685ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:43.787064 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb028151-07e4-4905-90e7-ba413cf5ba6b None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:43.787786 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb028151-07e4-4905-90e7-ba413cf5ba6b None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:43.788123 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb028151-07e4-4905-90e7-ba413cf5ba6b None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:43.790343 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 715/1427] 10.222.0.26 () {54 vars in 912 bytes} [Wed Nov 2 03:33:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:33:43.796392 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb78a05-4cb9-4cf6-9850-fb29b64459cb None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:43.796670 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb78a05-4cb9-4cf6-9850-fb29b64459cb None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:43.797094 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb78a05-4cb9-4cf6-9850-fb29b64459cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:43.817139 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-ef691fe5-eee9-49ea-a6ba-30d54b9685ce None None] MFA Rules not processed for user `eed64456ec5c42b08df447e9a537c4d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:43.818792 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ef691fe5-eee9-49ea-a6ba-30d54b9685ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:43.910106 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-6fb78a05-4cb9-4cf6-9850-fb29b64459cb None None] MFA Rules not processed for user `f07e1b51b00f42b7b74c3689268635cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:43.912792 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-6fb78a05-4cb9-4cf6-9850-fb29b64459cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:43.940969 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 716/1428] 10.222.0.26 () {58 vars in 1009 bytes} [Wed Nov 2 03:33:43 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:44.009578 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 713/1429] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:43 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:44.026601 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43650620-6b33-49ad-9b48-12cc002e9235 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:44.026944 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43650620-6b33-49ad-9b48-12cc002e9235 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:44.027224 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43650620-6b33-49ad-9b48-12cc002e9235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:44.076945 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-43650620-6b33-49ad-9b48-12cc002e9235 None None] MFA Rules not processed for user `eed64456ec5c42b08df447e9a537c4d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:44.103100 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-43650620-6b33-49ad-9b48-12cc002e9235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:44.124150 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 717/1430] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:44 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:44.133932 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef691fe5-eee9-49ea-a6ba-30d54b9685ce None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:44.152663 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef691fe5-eee9-49ea-a6ba-30d54b9685ce None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:44.164406 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a2be89-7c8d-4bc9-9f32-c0fd26ac6403 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:44.165736 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a2be89-7c8d-4bc9-9f32-c0fd26ac6403 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:44.168365 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a2be89-7c8d-4bc9-9f32-c0fd26ac6403 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:44.170523 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a2be89-7c8d-4bc9-9f32-c0fd26ac6403 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:44.205633 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a2be89-7c8d-4bc9-9f32-c0fd26ac6403 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:44.207514 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a2be89-7c8d-4bc9-9f32-c0fd26ac6403 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:44.220464 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 714/1431] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:33:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:33:44.678003 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43650620-6b33-49ad-9b48-12cc002e9235 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:44.680482 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-43650620-6b33-49ad-9b48-12cc002e9235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:44.704766 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43650620-6b33-49ad-9b48-12cc002e9235 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:44.718588 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76245522-dd3c-40e2-8a10-f60a2d5f5b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:44.719652 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76245522-dd3c-40e2-8a10-f60a2d5f5b11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:44.720146 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76245522-dd3c-40e2-8a10-f60a2d5f5b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:44.720625 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76245522-dd3c-40e2-8a10-f60a2d5f5b11 admin admin] PATH_INFO: `/v3/projects/94822d694220434bb08773106f1cb157` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:44.727106 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76245522-dd3c-40e2-8a10-f60a2d5f5b11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94822d694220434bb08773106f1cb157)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:44.729412 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76245522-dd3c-40e2-8a10-f60a2d5f5b11 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:44.812866 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f27c03-b369-40c2-80e3-105fe80984d4 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:44.813200 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f27c03-b369-40c2-80e3-105fe80984d4 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:44.813484 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f27c03-b369-40c2-80e3-105fe80984d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:44.876830 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-36f27c03-b369-40c2-80e3-105fe80984d4 None None] MFA Rules not processed for user `7984dc1d452a4c998b44e6d26a299df8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:44.878339 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-36f27c03-b369-40c2-80e3-105fe80984d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:44.881941 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-76245522-dd3c-40e2-8a10-f60a2d5f5b11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94822d694220434bb08773106f1cb157', 'request_id': 'req-76245522-dd3c-40e2-8a10-f60a2d5f5b11'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:44.898780 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-76245522-dd3c-40e2-8a10-f60a2d5f5b11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94822d694220434bb08773106f1cb157', 'request_id': 'req-76245522-dd3c-40e2-8a10-f60a2d5f5b11'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:44.908513 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-76245522-dd3c-40e2-8a10-f60a2d5f5b11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94822d694220434bb08773106f1cb157', 'request_id': 'req-76245522-dd3c-40e2-8a10-f60a2d5f5b11'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:44.965509 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-76245522-dd3c-40e2-8a10-f60a2d5f5b11 admin admin] The token cache is being invalidate because project 94822d694220434bb08773106f1cb157 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 03:33:44.966322 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-76245522-dd3c-40e2-8a10-f60a2d5f5b11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:44.969771 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-76245522-dd3c-40e2-8a10-f60a2d5f5b11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:44.981974 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 718/1432] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 03:33:44 2022] DELETE /identity/v3/projects/94822d694220434bb08773106f1cb157 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:44.984563 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db78d34a-98a1-4a5a-a2cb-b666078de905 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:44.986916 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db78d34a-98a1-4a5a-a2cb-b666078de905 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:44.988575 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db78d34a-98a1-4a5a-a2cb-b666078de905 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:44.993713 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 719/1433] 10.222.0.26 () {54 vars in 912 bytes} [Wed Nov 2 03:33:44 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:33:44.997137 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229fea8f-8a62-4ff0-bdfe-4a45f8a30540 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:44.999503 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229fea8f-8a62-4ff0-bdfe-4a45f8a30540 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:45.001831 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229fea8f-8a62-4ff0-bdfe-4a45f8a30540 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:45.026501 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 715/1434] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:45.028035 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f27c03-b369-40c2-80e3-105fe80984d4 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:45.032231 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-36f27c03-b369-40c2-80e3-105fe80984d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:45.065405 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-229fea8f-8a62-4ff0-bdfe-4a45f8a30540 None None] MFA Rules not processed for user `18355f4225c9427dbb38fde84115d7b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:45.067478 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-229fea8f-8a62-4ff0-bdfe-4a45f8a30540 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:45.096143 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:45.098681 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:45.174084 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f27c03-b369-40c2-80e3-105fe80984d4 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:45.194804 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5c0eca-721e-46a4-8f9f-07bd59064af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:45.195709 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c0eca-721e-46a4-8f9f-07bd59064af8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:45.196353 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c0eca-721e-46a4-8f9f-07bd59064af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:45.196353 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5c0eca-721e-46a4-8f9f-07bd59064af8 admin admin] PATH_INFO: `/v3/users/0fdff282e1f34e0e9267ad7164b351b5` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:45.197542 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5c0eca-721e-46a4-8f9f-07bd59064af8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fdff282e1f34e0e9267ad7164b351b5)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:45.199864 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5c0eca-721e-46a4-8f9f-07bd59064af8 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:45.216626 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 720/1435] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:45.228900 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bf2bd7-6bf0-4d69-97b7-5294e69fdbb7 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:45.228900 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bf2bd7-6bf0-4d69-97b7-5294e69fdbb7 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:45.229346 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bf2bd7-6bf0-4d69-97b7-5294e69fdbb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:45.301890 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-b2bf2bd7-6bf0-4d69-97b7-5294e69fdbb7 None None] MFA Rules not processed for user `f07e1b51b00f42b7b74c3689268635cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:45.308357 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-b2bf2bd7-6bf0-4d69-97b7-5294e69fdbb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:45.326576 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-fa5c0eca-721e-46a4-8f9f-07bd59064af8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fdff282e1f34e0e9267ad7164b351b5'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:45.352447 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-fa5c0eca-721e-46a4-8f9f-07bd59064af8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fdff282e1f34e0e9267ad7164b351b5'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:45.353221 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-fa5c0eca-721e-46a4-8f9f-07bd59064af8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fdff282e1f34e0e9267ad7164b351b5'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:45.382430 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-fa5c0eca-721e-46a4-8f9f-07bd59064af8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fdff282e1f34e0e9267ad7164b351b5'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:45.393560 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-fa5c0eca-721e-46a4-8f9f-07bd59064af8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fdff282e1f34e0e9267ad7164b351b5'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:45.401732 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 716/1436] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 03:33:45 2022] DELETE /identity/v3/users/0fdff282e1f34e0e9267ad7164b351b5 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:45.403254 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5c0eca-721e-46a4-8f9f-07bd59064af8 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:45.404999 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-fa5c0eca-721e-46a4-8f9f-07bd59064af8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:45.432224 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 721/1437] 10.222.0.26 () {58 vars in 1009 bytes} [Wed Nov 2 03:33:45 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:45.447139 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bf2bd7-6bf0-4d69-97b7-5294e69fdbb7 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:45.448842 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-b2bf2bd7-6bf0-4d69-97b7-5294e69fdbb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:45.466395 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bf2bd7-6bf0-4d69-97b7-5294e69fdbb7 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:45.537080 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5c0eca-721e-46a4-8f9f-07bd59064af8 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:45.539588 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cd5610-6c01-42b1-bd0a-8566caf44191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:45.540274 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd5610-6c01-42b1-bd0a-8566caf44191 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:45.540554 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd5610-6c01-42b1-bd0a-8566caf44191 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:45.540988 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd5610-6c01-42b1-bd0a-8566caf44191 admin admin] PATH_INFO: `/v3/projects/02805fc23cc54acf9fd68a8a28c41508` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:45.542531 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cd5610-6c01-42b1-bd0a-8566caf44191 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02805fc23cc54acf9fd68a8a28c41508)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:45.546442 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cd5610-6c01-42b1-bd0a-8566caf44191 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:45.626757 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c576d1-ebeb-468a-a7e9-ac6d490df7d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:45.627671 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c576d1-ebeb-468a-a7e9-ac6d490df7d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:45.627671 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c576d1-ebeb-468a-a7e9-ac6d490df7d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:45.627928 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c576d1-ebeb-468a-a7e9-ac6d490df7d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:45.630785 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:45.635534 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d9cd5610-6c01-42b1-bd0a-8566caf44191 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02805fc23cc54acf9fd68a8a28c41508', 'request_id': 'req-d9cd5610-6c01-42b1-bd0a-8566caf44191'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:45.646162 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c576d1-ebeb-468a-a7e9-ac6d490df7d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:45.650058 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c576d1-ebeb-468a-a7e9-ac6d490df7d3 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:45.654032 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d9cd5610-6c01-42b1-bd0a-8566caf44191 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02805fc23cc54acf9fd68a8a28c41508', 'request_id': 'req-d9cd5610-6c01-42b1-bd0a-8566caf44191'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:45.667111 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d9cd5610-6c01-42b1-bd0a-8566caf44191 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02805fc23cc54acf9fd68a8a28c41508', 'request_id': 'req-d9cd5610-6c01-42b1-bd0a-8566caf44191'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:45.679731 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 722/1438] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:33:45 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 03:33:45.683383 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c576d1-ebeb-468a-a7e9-ac6d490df7d3 service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:45.691454 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-08c576d1-ebeb-468a-a7e9-ac6d490df7d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:45.693192 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d9cd5610-6c01-42b1-bd0a-8566caf44191 admin admin] The token cache is being invalidate because project 02805fc23cc54acf9fd68a8a28c41508 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 03:33:45.693491 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d9cd5610-6c01-42b1-bd0a-8566caf44191 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:45.699057 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d9cd5610-6c01-42b1-bd0a-8566caf44191 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:45.700971 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 717/1439] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 03:33:45 2022] DELETE /identity/v3/projects/02805fc23cc54acf9fd68a8a28c41508 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:45.704864 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cd5610-6c01-42b1-bd0a-8566caf44191 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:45.707324 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d9cd5610-6c01-42b1-bd0a-8566caf44191 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:45.891296 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cd5610-6c01-42b1-bd0a-8566caf44191 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:45.893239 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c576d1-ebeb-468a-a7e9-ac6d490df7d3 service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:45.898039 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a6a883-d33f-46cf-bfe2-8ed9ec487305 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4b9bd741574ddabadd4b7651ebc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:45.898768 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:45.898880 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6a883-d33f-46cf-bfe2-8ed9ec487305 service placement] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:45.899307 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6a883-d33f-46cf-bfe2-8ed9ec487305 service placement] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:45.899645 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a6a883-d33f-46cf-bfe2-8ed9ec487305 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:45.902851 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a8a6a883-d33f-46cf-bfe2-8ed9ec487305 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:45.925845 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a6a883-d33f-46cf-bfe2-8ed9ec487305 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:45.928730 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a6a883-d33f-46cf-bfe2-8ed9ec487305 service placement] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:45.944512 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 718/1440] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:33:45 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:33:45.945683 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a6a883-d33f-46cf-bfe2-8ed9ec487305 service placement] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:45.947467 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a8a6a883-d33f-46cf-bfe2-8ed9ec487305 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:46.037431 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9efab8a-2cc5-4dc0-9176-08768633aad0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:46.038069 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9efab8a-2cc5-4dc0-9176-08768633aad0 service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:46.038347 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9efab8a-2cc5-4dc0-9176-08768633aad0 service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:46.038620 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9efab8a-2cc5-4dc0-9176-08768633aad0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:46.043355 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a9efab8a-2cc5-4dc0-9176-08768633aad0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:46.095166 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a6a883-d33f-46cf-bfe2-8ed9ec487305 service placement] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:46.109334 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c651685a-30b8-4a15-9ca7-82e52af316e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:46.112254 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651685a-30b8-4a15-9ca7-82e52af316e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:46.112642 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651685a-30b8-4a15-9ca7-82e52af316e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:46.113660 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c651685a-30b8-4a15-9ca7-82e52af316e1 admin admin] PATH_INFO: `/v3/users/b9a22711d03e418bb8323d0d341c2fb8` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:46.121815 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c651685a-30b8-4a15-9ca7-82e52af316e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9a22711d03e418bb8323d0d341c2fb8)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:46.133736 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c651685a-30b8-4a15-9ca7-82e52af316e1 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:46.212749 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9efab8a-2cc5-4dc0-9176-08768633aad0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:46.214600 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9efab8a-2cc5-4dc0-9176-08768633aad0 service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:46.238003 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 723/1441] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:33:45 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 555 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:33:46.239574 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9efab8a-2cc5-4dc0-9176-08768633aad0 service nova] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:46.241245 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a9efab8a-2cc5-4dc0-9176-08768633aad0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:46.258244 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9efab8a-2cc5-4dc0-9176-08768633aad0 service nova] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:46.270947 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8cfe30-07f8-473c-b8f0-3db62666dea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:46.271850 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8cfe30-07f8-473c-b8f0-3db62666dea1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:46.272383 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8cfe30-07f8-473c-b8f0-3db62666dea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:46.272723 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8cfe30-07f8-473c-b8f0-3db62666dea1 admin admin] PATH_INFO: `/v3/projects/f2b2a25d39154c75b0a0589327aba5d0` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:46.274431 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8cfe30-07f8-473c-b8f0-3db62666dea1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2b2a25d39154c75b0a0589327aba5d0)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:46.274962 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-c651685a-30b8-4a15-9ca7-82e52af316e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9a22711d03e418bb8323d0d341c2fb8'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:46.278165 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8cfe30-07f8-473c-b8f0-3db62666dea1 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:46.304751 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-c651685a-30b8-4a15-9ca7-82e52af316e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9a22711d03e418bb8323d0d341c2fb8'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:46.308895 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-c651685a-30b8-4a15-9ca7-82e52af316e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9a22711d03e418bb8323d0d341c2fb8'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:46.326092 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:46.344951 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-c651685a-30b8-4a15-9ca7-82e52af316e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9a22711d03e418bb8323d0d341c2fb8'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:46.360728 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-c651685a-30b8-4a15-9ca7-82e52af316e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9a22711d03e418bb8323d0d341c2fb8'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:46.388693 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 719/1442] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 03:33:45 2022] DELETE /identity/v3/users/b9a22711d03e418bb8323d0d341c2fb8 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:46.432954 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2c8cfe30-07f8-473c-b8f0-3db62666dea1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2b2a25d39154c75b0a0589327aba5d0', 'request_id': 'req-2c8cfe30-07f8-473c-b8f0-3db62666dea1'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:46.446631 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2c8cfe30-07f8-473c-b8f0-3db62666dea1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2b2a25d39154c75b0a0589327aba5d0', 'request_id': 'req-2c8cfe30-07f8-473c-b8f0-3db62666dea1'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:46.458485 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2c8cfe30-07f8-473c-b8f0-3db62666dea1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2b2a25d39154c75b0a0589327aba5d0', 'request_id': 'req-2c8cfe30-07f8-473c-b8f0-3db62666dea1'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:46.487441 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2c8cfe30-07f8-473c-b8f0-3db62666dea1 admin admin] The token cache is being invalidate because project f2b2a25d39154c75b0a0589327aba5d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 03:33:46.488022 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2c8cfe30-07f8-473c-b8f0-3db62666dea1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:46.490164 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2c8cfe30-07f8-473c-b8f0-3db62666dea1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:46.494518 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 724/1443] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 03:33:46 2022] DELETE /identity/v3/projects/f2b2a25d39154c75b0a0589327aba5d0 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:46.499599 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8c1749-a713-403a-b2e0-fe3cd0a16c6f None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:46.499890 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8c1749-a713-403a-b2e0-fe3cd0a16c6f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:46.500248 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8c1749-a713-403a-b2e0-fe3cd0a16c6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:46.546388 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-7c8c1749-a713-403a-b2e0-fe3cd0a16c6f None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:46.549750 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7c8c1749-a713-403a-b2e0-fe3cd0a16c6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:46.644254 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 720/1444] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:46.653512 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c4762-216d-4d64-875a-c1b750549240 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:46.653804 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c4762-216d-4d64-875a-c1b750549240 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:46.654082 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9c4762-216d-4d64-875a-c1b750549240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:46.683912 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-ff9c4762-216d-4d64-875a-c1b750549240 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:46.685307 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ff9c4762-216d-4d64-875a-c1b750549240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:46.701009 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8c1749-a713-403a-b2e0-fe3cd0a16c6f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:46.702341 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 725/1445] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:46.714448 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8c1749-a713-403a-b2e0-fe3cd0a16c6f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:46.724205 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04c9f10-a028-415f-87a5-23adb45b6358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:46.724205 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04c9f10-a028-415f-87a5-23adb45b6358 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:46.724205 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04c9f10-a028-415f-87a5-23adb45b6358 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:46.724205 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c04c9f10-a028-415f-87a5-23adb45b6358 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:46.725121 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04c9f10-a028-415f-87a5-23adb45b6358 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:33:46.725509 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04c9f10-a028-415f-87a5-23adb45b6358 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:46.728413 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c04c9f10-a028-415f-87a5-23adb45b6358 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:46.745490 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 721/1446] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 03:33:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:33:46.750106 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9c4762-216d-4d64-875a-c1b750549240 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:46.764315 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9c4762-216d-4d64-875a-c1b750549240 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:46.772292 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96666d32-39bd-4c96-848e-e6769a34bc84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:46.773087 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96666d32-39bd-4c96-848e-e6769a34bc84 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:46.773619 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96666d32-39bd-4c96-848e-e6769a34bc84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:46.774336 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96666d32-39bd-4c96-848e-e6769a34bc84 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:46.775791 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96666d32-39bd-4c96-848e-e6769a34bc84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:46.778069 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96666d32-39bd-4c96-848e-e6769a34bc84 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:46.825123 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:46.835911 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 726/1447] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:33:46 2022] POST /identity/v3/projects => generated 383 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 03:33:46.839776 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04c9f10-a028-415f-87a5-23adb45b6358 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:46.858517 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c04c9f10-a028-415f-87a5-23adb45b6358 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:47.072794 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddd1f04-46f0-440c-b271-7195f2e0ffe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:47.073431 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddd1f04-46f0-440c-b271-7195f2e0ffe6 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:47.073712 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddd1f04-46f0-440c-b271-7195f2e0ffe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:47.076636 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddd1f04-46f0-440c-b271-7195f2e0ffe6 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:47.077396 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddd1f04-46f0-440c-b271-7195f2e0ffe6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:47.080165 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddd1f04-46f0-440c-b271-7195f2e0ffe6 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:47.148857 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 722/1448] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:33:46 2022] POST /identity/v3/users => generated 292 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:33:47.153188 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96666d32-39bd-4c96-848e-e6769a34bc84 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:47.171282 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96666d32-39bd-4c96-848e-e6769a34bc84 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:47.189792 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb94bf5-bfc4-4647-837b-8d8df395cbb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:47.190501 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb94bf5-bfc4-4647-837b-8d8df395cbb0 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:47.190790 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb94bf5-bfc4-4647-837b-8d8df395cbb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:47.191059 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb94bf5-bfc4-4647-837b-8d8df395cbb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:47.191722 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb94bf5-bfc4-4647-837b-8d8df395cbb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:47.195917 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb94bf5-bfc4-4647-837b-8d8df395cbb0 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:47.235068 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 727/1449] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:33:47 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:33:47.239689 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddd1f04-46f0-440c-b271-7195f2e0ffe6 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:47.254360 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddd1f04-46f0-440c-b271-7195f2e0ffe6 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:47.269313 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73803ba-6769-4865-9459-cfbd67691539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:47.270054 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73803ba-6769-4865-9459-cfbd67691539 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:47.270419 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73803ba-6769-4865-9459-cfbd67691539 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:47.270767 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73803ba-6769-4865-9459-cfbd67691539 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:47.271496 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73803ba-6769-4865-9459-cfbd67691539 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:47.275478 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73803ba-6769-4865-9459-cfbd67691539 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:47.292606 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 723/1450] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:33:47 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:33:47.296892 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb94bf5-bfc4-4647-837b-8d8df395cbb0 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:47.316140 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb94bf5-bfc4-4647-837b-8d8df395cbb0 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:47.331268 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baeb360-dc1e-4407-9720-2af9f007b0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:47.332079 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baeb360-dc1e-4407-9720-2af9f007b0a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:47.332499 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baeb360-dc1e-4407-9720-2af9f007b0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:47.332864 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0baeb360-dc1e-4407-9720-2af9f007b0a9 admin admin] PATH_INFO: `/v3/projects/f9f3451911fe43f596301e39fc0f6593/users/226e91def5844ba08fb5108d824d7f78/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:47.372096 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baeb360-dc1e-4407-9720-2af9f007b0a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9f3451911fe43f596301e39fc0f6593, user_id=226e91def5844ba08fb5108d824d7f78, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:47.375395 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0baeb360-dc1e-4407-9720-2af9f007b0a9 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:47.424294 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 728/1451] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:33:47 2022] PUT /identity/v3/projects/f9f3451911fe43f596301e39fc0f6593/users/226e91def5844ba08fb5108d824d7f78/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:47.436084 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e9d68-d1c8-4d94-9c7e-fdde0051a02d None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:47.436588 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e9d68-d1c8-4d94-9c7e-fdde0051a02d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:47.437084 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0e9d68-d1c8-4d94-9c7e-fdde0051a02d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:47.466993 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baeb360-dc1e-4407-9720-2af9f007b0a9 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:47.469240 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-0baeb360-dc1e-4407-9720-2af9f007b0a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:47.543883 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-8e0e9d68-d1c8-4d94-9c7e-fdde0051a02d None None] MFA Rules not processed for user `226e91def5844ba08fb5108d824d7f78`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:47.546247 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8e0e9d68-d1c8-4d94-9c7e-fdde0051a02d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:47.734030 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:47.747078 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0baeb360-dc1e-4407-9720-2af9f007b0a9 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:47.785164 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f292a3-8984-4646-9daf-4a68bced2466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:47.786077 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f292a3-8984-4646-9daf-4a68bced2466 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:47.786521 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f292a3-8984-4646-9daf-4a68bced2466 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:47.786933 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f292a3-8984-4646-9daf-4a68bced2466 admin admin] PATH_INFO: `/v3/projects/d67a3c547637490097cb05d74d62383c` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:47.788732 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f292a3-8984-4646-9daf-4a68bced2466 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d67a3c547637490097cb05d74d62383c)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:47.790793 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f292a3-8984-4646-9daf-4a68bced2466 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:47.834785 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 724/1452] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 400 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:47.835780 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0e9d68-d1c8-4d94-9c7e-fdde0051a02d None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:47.839074 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8e0e9d68-d1c8-4d94-9c7e-fdde0051a02d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:47.952985 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b8f292a3-8984-4646-9daf-4a68bced2466 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd67a3c547637490097cb05d74d62383c', 'request_id': 'req-b8f292a3-8984-4646-9daf-4a68bced2466'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:47.969588 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b8f292a3-8984-4646-9daf-4a68bced2466 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd67a3c547637490097cb05d74d62383c', 'request_id': 'req-b8f292a3-8984-4646-9daf-4a68bced2466'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:47.977844 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b8f292a3-8984-4646-9daf-4a68bced2466 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd67a3c547637490097cb05d74d62383c', 'request_id': 'req-b8f292a3-8984-4646-9daf-4a68bced2466'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:48.015274 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b8f292a3-8984-4646-9daf-4a68bced2466 admin admin] The token cache is being invalidate because project d67a3c547637490097cb05d74d62383c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 03:33:48.017155 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b8f292a3-8984-4646-9daf-4a68bced2466 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:48.019640 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b8f292a3-8984-4646-9daf-4a68bced2466 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:48.025748 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 729/1453] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 03:33:47 2022] DELETE /identity/v3/projects/d67a3c547637490097cb05d74d62383c => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:48.026819 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f292a3-8984-4646-9daf-4a68bced2466 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:48.033831 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-b8f292a3-8984-4646-9daf-4a68bced2466 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:48.066481 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0e9d68-d1c8-4d94-9c7e-fdde0051a02d None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:48.204918 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:48.217926 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f292a3-8984-4646-9daf-4a68bced2466 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:48.225406 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f8cc62-b222-4d93-a8b1-bf5ccb02647a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:48.226168 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f8cc62-b222-4d93-a8b1-bf5ccb02647a service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:48.226399 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f8cc62-b222-4d93-a8b1-bf5ccb02647a service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:48.226766 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f8cc62-b222-4d93-a8b1-bf5ccb02647a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:48.228809 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c9f8cc62-b222-4d93-a8b1-bf5ccb02647a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:48.237269 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0043c5-e7cc-4411-bc5d-3dbe9b772ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:48.238330 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0043c5-e7cc-4411-bc5d-3dbe9b772ad7 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:48.238330 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0043c5-e7cc-4411-bc5d-3dbe9b772ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:48.238490 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0043c5-e7cc-4411-bc5d-3dbe9b772ad7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:48.239075 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0043c5-e7cc-4411-bc5d-3dbe9b772ad7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:48.241100 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0043c5-e7cc-4411-bc5d-3dbe9b772ad7 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:48.311567 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 730/1454] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:33:48 2022] POST /identity/v3/projects => generated 383 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:33:48.318935 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10599153-77a2-48b2-bb27-6b8e8f1191f4 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:48.320177 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10599153-77a2-48b2-bb27-6b8e8f1191f4 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:48.320640 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10599153-77a2-48b2-bb27-6b8e8f1191f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:48.390353 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-10599153-77a2-48b2-bb27-6b8e8f1191f4 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:48.393053 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-10599153-77a2-48b2-bb27-6b8e8f1191f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:48.414535 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f8cc62-b222-4d93-a8b1-bf5ccb02647a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:48.416592 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f8cc62-b222-4d93-a8b1-bf5ccb02647a service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:48.429228 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 725/1455] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:33:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 594 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:33:48.430073 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f8cc62-b222-4d93-a8b1-bf5ccb02647a service nova] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:48.440968 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f8cc62-b222-4d93-a8b1-bf5ccb02647a service nova] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:48.480763 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:48.679542 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 731/1456] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:48.687119 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d407c633-0407-4d16-95d1-e36eaeb11089 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:48.687725 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d407c633-0407-4d16-95d1-e36eaeb11089 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:48.688137 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d407c633-0407-4d16-95d1-e36eaeb11089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:48.721825 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c69683e-6c01-4527-a6ae-34fa2b9cfe7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:48.729973 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c69683e-6c01-4527-a6ae-34fa2b9cfe7a admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:48.729973 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c69683e-6c01-4527-a6ae-34fa2b9cfe7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:48.729973 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c69683e-6c01-4527-a6ae-34fa2b9cfe7a admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:48.729973 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c69683e-6c01-4527-a6ae-34fa2b9cfe7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:48.729973 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c69683e-6c01-4527-a6ae-34fa2b9cfe7a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:48.783795 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-d407c633-0407-4d16-95d1-e36eaeb11089 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:48.791380 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d407c633-0407-4d16-95d1-e36eaeb11089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:48.829825 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 726/1457] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:33:48 2022] POST /identity/v3/users => generated 298 bytes in 397 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:33:48.831110 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c69683e-6c01-4527-a6ae-34fa2b9cfe7a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:48.835007 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 732/1458] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:48.839375 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d407c633-0407-4d16-95d1-e36eaeb11089 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:48.859495 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c69683e-6c01-4527-a6ae-34fa2b9cfe7a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:48.876818 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d407c633-0407-4d16-95d1-e36eaeb11089 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:48.877957 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2974e931-5a69-42e5-b7f7-c0213cdd681f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:48.880487 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2974e931-5a69-42e5-b7f7-c0213cdd681f admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:48.880894 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2974e931-5a69-42e5-b7f7-c0213cdd681f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:48.881728 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2974e931-5a69-42e5-b7f7-c0213cdd681f admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:48.885619 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2974e931-5a69-42e5-b7f7-c0213cdd681f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:48.889567 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2974e931-5a69-42e5-b7f7-c0213cdd681f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:48.894828 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf83c33-bf24-4a7f-860e-ded4f85eba59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:48.895474 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf83c33-bf24-4a7f-860e-ded4f85eba59 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:48.895750 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf83c33-bf24-4a7f-860e-ded4f85eba59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:48.896049 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf83c33-bf24-4a7f-860e-ded4f85eba59 admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:48.896854 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf83c33-bf24-4a7f-860e-ded4f85eba59 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:33:48.899765 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf83c33-bf24-4a7f-860e-ded4f85eba59 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:48.905659 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf83c33-bf24-4a7f-860e-ded4f85eba59 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:48.941453 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 727/1459] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:33:48 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:33:48.945203 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2974e931-5a69-42e5-b7f7-c0213cdd681f admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:48.952289 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 733/1460] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 03:33:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:33:48.956925 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf83c33-bf24-4a7f-860e-ded4f85eba59 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:49.000400 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf83c33-bf24-4a7f-860e-ded4f85eba59 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:49.002485 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2974e931-5a69-42e5-b7f7-c0213cdd681f admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:49.018716 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc84961-4765-4e53-b4fd-289f7b136163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:49.019940 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc84961-4765-4e53-b4fd-289f7b136163 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:49.020582 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc84961-4765-4e53-b4fd-289f7b136163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:49.021045 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc84961-4765-4e53-b4fd-289f7b136163 admin admin] PATH_INFO: `/v3/projects/64dfeff7c30e49f68600f1f490976e85/users/f15d53e7dec2486985d031184ba4f3fa/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:49.023797 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94beaeb-dd2d-439a-bf52-8f3eed4bbf10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:49.027267 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94beaeb-dd2d-439a-bf52-8f3eed4bbf10 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:49.027569 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94beaeb-dd2d-439a-bf52-8f3eed4bbf10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:49.028810 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94beaeb-dd2d-439a-bf52-8f3eed4bbf10 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:49.033181 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94beaeb-dd2d-439a-bf52-8f3eed4bbf10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:49.042230 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94beaeb-dd2d-439a-bf52-8f3eed4bbf10 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:49.047850 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc84961-4765-4e53-b4fd-289f7b136163 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64dfeff7c30e49f68600f1f490976e85, user_id=f15d53e7dec2486985d031184ba4f3fa, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:49.051854 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc84961-4765-4e53-b4fd-289f7b136163 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:49.129930 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 728/1461] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:33:48 2022] PUT /identity/v3/projects/64dfeff7c30e49f68600f1f490976e85/users/f15d53e7dec2486985d031184ba4f3fa/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:49.134214 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc84961-4765-4e53-b4fd-289f7b136163 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:49.197794 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 734/1462] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:33:48 2022] POST /identity/v3/projects => generated 377 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:33:49.203699 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94beaeb-dd2d-439a-bf52-8f3eed4bbf10 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:49.393693 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc84961-4765-4e53-b4fd-289f7b136163 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:49.458192 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94beaeb-dd2d-439a-bf52-8f3eed4bbf10 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:49.464623 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf77b95-c045-4aaf-8ed8-861054311ad7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:49.465392 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf77b95-c045-4aaf-8ed8-861054311ad7 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:49.465760 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf77b95-c045-4aaf-8ed8-861054311ad7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:49.466379 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf77b95-c045-4aaf-8ed8-861054311ad7 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:49.467506 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf77b95-c045-4aaf-8ed8-861054311ad7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:49.471065 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf77b95-c045-4aaf-8ed8-861054311ad7 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:49.527407 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 735/1463] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:33:49 2022] POST /identity/v3/users => generated 289 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:33:49.530949 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf77b95-c045-4aaf-8ed8-861054311ad7 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:49.586901 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf77b95-c045-4aaf-8ed8-861054311ad7 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:49.603132 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceccf7e4-64db-4fd4-ae67-40421289e4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:49.605327 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceccf7e4-64db-4fd4-ae67-40421289e4dc admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:49.609130 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceccf7e4-64db-4fd4-ae67-40421289e4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:49.609574 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceccf7e4-64db-4fd4-ae67-40421289e4dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:49.612380 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceccf7e4-64db-4fd4-ae67-40421289e4dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:49.616819 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceccf7e4-64db-4fd4-ae67-40421289e4dc admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:49.662166 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 736/1464] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:33:49 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:33:49.668375 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceccf7e4-64db-4fd4-ae67-40421289e4dc admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:49.692559 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac496d5d-a175-4419-9492-3a8c786c7d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:49.694136 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac496d5d-a175-4419-9492-3a8c786c7d8c admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:49.697173 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac496d5d-a175-4419-9492-3a8c786c7d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:49.698150 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac496d5d-a175-4419-9492-3a8c786c7d8c admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:49.700707 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac496d5d-a175-4419-9492-3a8c786c7d8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:49.718779 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceccf7e4-64db-4fd4-ae67-40421289e4dc admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:49.721300 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac496d5d-a175-4419-9492-3a8c786c7d8c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:49.751921 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4be436-f1b7-46ec-9f03-b4f32f92f0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:49.752678 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4be436-f1b7-46ec-9f03-b4f32f92f0e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:49.759365 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 729/1465] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:33:49 2022] GET /identity/v3/roles => generated 1420 bytes in 623 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:33:49.759573 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4be436-f1b7-46ec-9f03-b4f32f92f0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:49.759666 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c53af5c-b95c-4ce4-9037-24f5a426cd2a None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:49.759666 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c53af5c-b95c-4ce4-9037-24f5a426cd2a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:49.759666 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c53af5c-b95c-4ce4-9037-24f5a426cd2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:49.761834 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4be436-f1b7-46ec-9f03-b4f32f92f0e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:49.763426 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4be436-f1b7-46ec-9f03-b4f32f92f0e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:49.774664 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4be436-f1b7-46ec-9f03-b4f32f92f0e9 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:49.811361 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-9c53af5c-b95c-4ce4-9037-24f5a426cd2a None None] MFA Rules not processed for user `eed64456ec5c42b08df447e9a537c4d0`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:49.811903 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 737/1466] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:33:49 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:33:49.813613 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9c53af5c-b95c-4ce4-9037-24f5a426cd2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:49.814287 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4be436-f1b7-46ec-9f03-b4f32f92f0e9 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:49.826910 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:49.837043 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4be436-f1b7-46ec-9f03-b4f32f92f0e9 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:49.863757 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6829b340-90c4-468b-b8c1-ae2de39934db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:49.864639 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6829b340-90c4-468b-b8c1-ae2de39934db admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:49.865021 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6829b340-90c4-468b-b8c1-ae2de39934db admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:49.867372 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6829b340-90c4-468b-b8c1-ae2de39934db admin admin] PATH_INFO: `/v3/projects/64dfeff7c30e49f68600f1f490976e85/users/f15d53e7dec2486985d031184ba4f3fa/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:49.871580 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6829b340-90c4-468b-b8c1-ae2de39934db admin admin] RBAC: Authorizing `identity:create_grant(project_id=64dfeff7c30e49f68600f1f490976e85, user_id=f15d53e7dec2486985d031184ba4f3fa, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:49.879509 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6829b340-90c4-468b-b8c1-ae2de39934db admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:49.892007 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:49.913367 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 738/1467] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:33:49 2022] PUT /identity/v3/projects/64dfeff7c30e49f68600f1f490976e85/users/f15d53e7dec2486985d031184ba4f3fa/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:49.924020 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6829b340-90c4-468b-b8c1-ae2de39934db admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:50.059395 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 730/1468] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:49 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 300 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:50.089643 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5837933-af93-4b26-b909-6dcdbe41ae3c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:50.090097 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5837933-af93-4b26-b909-6dcdbe41ae3c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:50.090474 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5837933-af93-4b26-b909-6dcdbe41ae3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:50.178419 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-f5837933-af93-4b26-b909-6dcdbe41ae3c None None] MFA Rules not processed for user `f15d53e7dec2486985d031184ba4f3fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:50.181297 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f5837933-af93-4b26-b909-6dcdbe41ae3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:50.259930 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6829b340-90c4-468b-b8c1-ae2de39934db admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:50.292375 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d337b4-563c-46ce-9a3b-c473b2696ea8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:50.293312 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d337b4-563c-46ce-9a3b-c473b2696ea8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:50.293705 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d337b4-563c-46ce-9a3b-c473b2696ea8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:50.293992 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d337b4-563c-46ce-9a3b-c473b2696ea8 admin admin] PATH_INFO: `/v3/projects/ccc96c73927a495b9e183382a7252fa5/users/8b3d710f203542158ab1fbba3d65c720/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:50.371798 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d337b4-563c-46ce-9a3b-c473b2696ea8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccc96c73927a495b9e183382a7252fa5, user_id=8b3d710f203542158ab1fbba3d65c720, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:50.375122 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d337b4-563c-46ce-9a3b-c473b2696ea8 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:50.428961 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 739/1469] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:33:49 2022] PUT /identity/v3/projects/ccc96c73927a495b9e183382a7252fa5/users/8b3d710f203542158ab1fbba3d65c720/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:50.439723 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 731/1470] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:50 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:50.440674 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370950e0-cb7d-4c99-9417-cbdf9811f0ee None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:50.441331 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370950e0-cb7d-4c99-9417-cbdf9811f0ee None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:50.441331 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370950e0-cb7d-4c99-9417-cbdf9811f0ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:50.463985 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5837933-af93-4b26-b909-6dcdbe41ae3c None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:50.467461 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f5837933-af93-4b26-b909-6dcdbe41ae3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:50.581149 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-370950e0-cb7d-4c99-9417-cbdf9811f0ee None None] MFA Rules not processed for user `8b3d710f203542158ab1fbba3d65c720`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:50.582417 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-370950e0-cb7d-4c99-9417-cbdf9811f0ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:50.720094 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5837933-af93-4b26-b909-6dcdbe41ae3c None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:50.885598 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 740/1471] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:50 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 446 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:50.890642 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e064797-0439-4b23-8cee-3d5ccce4db54 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:50.890932 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e064797-0439-4b23-8cee-3d5ccce4db54 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:50.891208 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e064797-0439-4b23-8cee-3d5ccce4db54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:50.908337 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f4bc01-00af-4924-accd-16df8f608e33 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:50.909133 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4bc01-00af-4924-accd-16df8f608e33 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:50.909537 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4bc01-00af-4924-accd-16df8f608e33 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:50.909919 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f4bc01-00af-4924-accd-16df8f608e33 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:50.949462 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:51.053012 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-5e064797-0439-4b23-8cee-3d5ccce4db54 None None] MFA Rules not processed for user `da7ee607b66e43f3823d9d2faf373974`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:51.054260 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-5e064797-0439-4b23-8cee-3d5ccce4db54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:51.156313 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:51.204362 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 741/1472] 10.222.0.26 () {58 vars in 1009 bytes} [Wed Nov 2 03:33:50 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:51.205222 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e064797-0439-4b23-8cee-3d5ccce4db54 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:51.401327 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f4bc01-00af-4924-accd-16df8f608e33 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:51.403035 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f4bc01-00af-4924-accd-16df8f608e33 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:51.422077 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e064797-0439-4b23-8cee-3d5ccce4db54 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:51.426354 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 732/1473] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:33:50 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 963 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:33:51.435259 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb4f25d-1543-4ee7-8819-509cbf59dd7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:51.436150 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb4f25d-1543-4ee7-8819-509cbf59dd7e admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:51.436540 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb4f25d-1543-4ee7-8819-509cbf59dd7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:51.436914 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb4f25d-1543-4ee7-8819-509cbf59dd7e admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:51.437982 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb4f25d-1543-4ee7-8819-509cbf59dd7e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:51.439854 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb4f25d-1543-4ee7-8819-509cbf59dd7e admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:51.523448 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 742/1474] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:33:51 2022] POST /identity/v3/projects => generated 381 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:33:51.528258 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f4bc01-00af-4924-accd-16df8f608e33 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:51.544699 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f4bc01-00af-4924-accd-16df8f608e33 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:51.583169 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb4f25d-1543-4ee7-8819-509cbf59dd7e admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:51.585095 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-cdb4f25d-1543-4ee7-8819-509cbf59dd7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:51.693282 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644a182b-fd36-4018-9037-f590da23e38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:51.694077 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644a182b-fd36-4018-9037-f590da23e38b admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:51.694454 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644a182b-fd36-4018-9037-f590da23e38b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:51.694828 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644a182b-fd36-4018-9037-f590da23e38b admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:51.696487 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644a182b-fd36-4018-9037-f590da23e38b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:51.700265 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644a182b-fd36-4018-9037-f590da23e38b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:51.765465 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 733/1475] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:33:51 2022] POST /identity/v3/users => generated 297 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:33:51.768574 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644a182b-fd36-4018-9037-f590da23e38b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:51.852877 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb4f25d-1543-4ee7-8819-509cbf59dd7e admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:51.868980 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f744406-54b2-4fbe-8a33-4aa7aad5b26f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5193c40a917b4105ab8b27aead1685f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:51.869885 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f744406-54b2-4fbe-8a33-4aa7aad5b26f service swift] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:51.871508 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f744406-54b2-4fbe-8a33-4aa7aad5b26f service swift] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:51.872021 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f744406-54b2-4fbe-8a33-4aa7aad5b26f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:51.949130 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644a182b-fd36-4018-9037-f590da23e38b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:51.963423 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40708cb2-838c-41c5-8e7c-7735ae226439 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:51.964475 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40708cb2-838c-41c5-8e7c-7735ae226439 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:51.965224 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40708cb2-838c-41c5-8e7c-7735ae226439 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:51.965709 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40708cb2-838c-41c5-8e7c-7735ae226439 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:51.972453 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-40708cb2-838c-41c5-8e7c-7735ae226439 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:51.997840 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:52.026913 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f744406-54b2-4fbe-8a33-4aa7aad5b26f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:52.028711 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f744406-54b2-4fbe-8a33-4aa7aad5b26f service swift] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:52.049100 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 743/1476] 10.222.0.26 () {60 vars in 1384 bytes} [Wed Nov 2 03:33:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Nov 02 03:33:52.052835 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f744406-54b2-4fbe-8a33-4aa7aad5b26f service swift] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:52.111725 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f744406-54b2-4fbe-8a33-4aa7aad5b26f service swift] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:52.114094 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:52.132764 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2019a773-c1b9-4a7f-8f2d-4643e0d5e168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:52.133407 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2019a773-c1b9-4a7f-8f2d-4643e0d5e168 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:52.133842 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2019a773-c1b9-4a7f-8f2d-4643e0d5e168 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:52.134282 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2019a773-c1b9-4a7f-8f2d-4643e0d5e168 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:52.135076 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2019a773-c1b9-4a7f-8f2d-4643e0d5e168 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:52.136899 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2019a773-c1b9-4a7f-8f2d-4643e0d5e168 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:52.152616 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40708cb2-838c-41c5-8e7c-7735ae226439 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:52.154072 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40708cb2-838c-41c5-8e7c-7735ae226439 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:52.175754 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 744/1477] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:33:52.180208 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2019a773-c1b9-4a7f-8f2d-4643e0d5e168 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:52.181406 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 734/1478] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:33:51 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:33:52.197900 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2019a773-c1b9-4a7f-8f2d-4643e0d5e168 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:52.215386 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff2c787-9977-408d-8806-bfb1d21e70a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:52.216527 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff2c787-9977-408d-8806-bfb1d21e70a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:52.217056 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff2c787-9977-408d-8806-bfb1d21e70a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:52.217474 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff2c787-9977-408d-8806-bfb1d21e70a0 admin admin] PATH_INFO: `/v3/projects/5cddc0bafebf45db8e4bb064deae81bd/users/170d9d09007d4feeadcb18b0befc1165/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:52.258501 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff2c787-9977-408d-8806-bfb1d21e70a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cddc0bafebf45db8e4bb064deae81bd, user_id=170d9d09007d4feeadcb18b0befc1165, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:52.269630 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff2c787-9977-408d-8806-bfb1d21e70a0 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:52.307352 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 745/1479] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:33:52 2022] PUT /identity/v3/projects/5cddc0bafebf45db8e4bb064deae81bd/users/170d9d09007d4feeadcb18b0befc1165/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:52.311605 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40708cb2-838c-41c5-8e7c-7735ae226439 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:52.327449 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40708cb2-838c-41c5-8e7c-7735ae226439 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:52.379221 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd7162-6316-4bcd-98e2-6f06630097b2 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:52.379686 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd7162-6316-4bcd-98e2-6f06630097b2 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:52.380146 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd7162-6316-4bcd-98e2-6f06630097b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:52.445367 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-8cfd7162-6316-4bcd-98e2-6f06630097b2 None None] MFA Rules not processed for user `18355f4225c9427dbb38fde84115d7b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:52.446256 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-8cfd7162-6316-4bcd-98e2-6f06630097b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:52.499142 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2a7825-04d0-496f-a5dc-d347b832c7c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:52.499791 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2a7825-04d0-496f-a5dc-d347b832c7c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:52.500114 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2a7825-04d0-496f-a5dc-d347b832c7c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:52.500637 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2a7825-04d0-496f-a5dc-d347b832c7c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:52.503035 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2a7825-04d0-496f-a5dc-d347b832c7c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:52.505782 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2a7825-04d0-496f-a5dc-d347b832c7c3 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:52.525719 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 735/1480] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 03:33:52.530844 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2a7825-04d0-496f-a5dc-d347b832c7c3 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:52.559498 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2a7825-04d0-496f-a5dc-d347b832c7c3 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:52.574894 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ecdb35-b998-4ef3-9d34-dc4928442ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:52.577001 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ecdb35-b998-4ef3-9d34-dc4928442ceb admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:52.578062 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ecdb35-b998-4ef3-9d34-dc4928442ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:52.579589 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ecdb35-b998-4ef3-9d34-dc4928442ceb admin admin] PATH_INFO: `/v3/projects/5cddc0bafebf45db8e4bb064deae81bd/users/170d9d09007d4feeadcb18b0befc1165/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:52.587028 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ecdb35-b998-4ef3-9d34-dc4928442ceb admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cddc0bafebf45db8e4bb064deae81bd, user_id=170d9d09007d4feeadcb18b0befc1165, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:52.590903 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ecdb35-b998-4ef3-9d34-dc4928442ceb admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:52.591377 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 746/1481] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 03:33:52.619615 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 736/1482] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:33:52 2022] PUT /identity/v3/projects/5cddc0bafebf45db8e4bb064deae81bd/users/170d9d09007d4feeadcb18b0befc1165/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:52.622055 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1088d0a4-fdbb-449d-bb05-668c3f2602d9 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:52.622913 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1088d0a4-fdbb-449d-bb05-668c3f2602d9 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:52.622913 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1088d0a4-fdbb-449d-bb05-668c3f2602d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:52.689881 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-1088d0a4-fdbb-449d-bb05-668c3f2602d9 None None] MFA Rules not processed for user `170d9d09007d4feeadcb18b0befc1165`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:52.694458 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-1088d0a4-fdbb-449d-bb05-668c3f2602d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:52.724542 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ecdb35-b998-4ef3-9d34-dc4928442ceb admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:52.882889 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ecdb35-b998-4ef3-9d34-dc4928442ceb admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:52.893965 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e9abde-8577-44a4-a897-a42d9694ae81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:52.894635 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e9abde-8577-44a4-a897-a42d9694ae81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:52.894914 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e9abde-8577-44a4-a897-a42d9694ae81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:52.898472 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 747/1483] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:52 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:52.901998 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e9abde-8577-44a4-a897-a42d9694ae81 admin admin] PATH_INFO: `/v3/users/226e91def5844ba08fb5108d824d7f78` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:52.909045 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e9abde-8577-44a4-a897-a42d9694ae81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=226e91def5844ba08fb5108d824d7f78)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:52.910598 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e9abde-8577-44a4-a897-a42d9694ae81 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:52.915733 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1088d0a4-fdbb-449d-bb05-668c3f2602d9 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:52.918703 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-1088d0a4-fdbb-449d-bb05-668c3f2602d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:53.042220 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:53.044820 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b7e9abde-8577-44a4-a897-a42d9694ae81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '226e91def5844ba08fb5108d824d7f78'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:53.064995 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b7e9abde-8577-44a4-a897-a42d9694ae81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '226e91def5844ba08fb5108d824d7f78'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:53.065753 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b7e9abde-8577-44a4-a897-a42d9694ae81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '226e91def5844ba08fb5108d824d7f78'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:53.082142 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1088d0a4-fdbb-449d-bb05-668c3f2602d9 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:53.093643 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd9b877-3f9c-4d1c-b015-d67f7c38afac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:53.094908 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd9b877-3f9c-4d1c-b015-d67f7c38afac service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:53.095202 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd9b877-3f9c-4d1c-b015-d67f7c38afac service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:53.095533 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd9b877-3f9c-4d1c-b015-d67f7c38afac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:53.103237 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b7e9abde-8577-44a4-a897-a42d9694ae81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '226e91def5844ba08fb5108d824d7f78'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:53.113169 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:INSERT=2,keystone:DELETE=9 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:53.121501 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b7e9abde-8577-44a4-a897-a42d9694ae81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '226e91def5844ba08fb5108d824d7f78'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:53.131823 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 737/1484] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 03:33:52 2022] DELETE /identity/v3/users/226e91def5844ba08fb5108d824d7f78 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 03:33:53.136434 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc5b45-fbbb-4710-8ecb-abadd3551de2 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:53.136982 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc5b45-fbbb-4710-8ecb-abadd3551de2 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:53.137393 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbc5b45-fbbb-4710-8ecb-abadd3551de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:53.138145 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd9b877-3f9c-4d1c-b015-d67f7c38afac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:53.142472 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd9b877-3f9c-4d1c-b015-d67f7c38afac service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:53.162964 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 748/1485] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:33:52 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 03:33:53.171286 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-5bbc5b45-fbbb-4710-8ecb-abadd3551de2 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:53.173257 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5bbc5b45-fbbb-4710-8ecb-abadd3551de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:53.364457 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 738/1486] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 03:33:53.380803 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd9b877-3f9c-4d1c-b015-d67f7c38afac service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:53.382535 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-0fd9b877-3f9c-4d1c-b015-d67f7c38afac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:53.474897 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd9b877-3f9c-4d1c-b015-d67f7c38afac service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:53.577512 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e4e960-4562-4719-b302-fa592d7b3013 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:53.578686 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e4e960-4562-4719-b302-fa592d7b3013 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:53.579413 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e4e960-4562-4719-b302-fa592d7b3013 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:53.580433 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e4e960-4562-4719-b302-fa592d7b3013 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:53.612975 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e4e960-4562-4719-b302-fa592d7b3013 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:53.615060 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e4e960-4562-4719-b302-fa592d7b3013 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:53.633174 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 749/1487] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:33:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:33:54.131338 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc5b45-fbbb-4710-8ecb-abadd3551de2 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:54.133928 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5bbc5b45-fbbb-4710-8ecb-abadd3551de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:54.166967 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbc5b45-fbbb-4710-8ecb-abadd3551de2 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:54.190905 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed61292d-a05a-4e87-9b12-06e910cd380a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:54.191728 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed61292d-a05a-4e87-9b12-06e910cd380a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:54.192093 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed61292d-a05a-4e87-9b12-06e910cd380a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:54.192474 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed61292d-a05a-4e87-9b12-06e910cd380a admin admin] PATH_INFO: `/v3/users/8b3d710f203542158ab1fbba3d65c720` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:54.194153 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed61292d-a05a-4e87-9b12-06e910cd380a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b3d710f203542158ab1fbba3d65c720)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:54.199345 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed61292d-a05a-4e87-9b12-06e910cd380a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:54.241584 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e4e960-4562-4719-b302-fa592d7b3013 service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:54.243994 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-78e4e960-4562-4719-b302-fa592d7b3013 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:54.287635 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e4e960-4562-4719-b302-fa592d7b3013 service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:54.316952 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d879e3b0-2258-4cd2-a995-e29e196396ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:54.317869 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d879e3b0-2258-4cd2-a995-e29e196396ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:54.317869 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d879e3b0-2258-4cd2-a995-e29e196396ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:54.318169 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d879e3b0-2258-4cd2-a995-e29e196396ab admin admin] PATH_INFO: `/v3/projects/f9f3451911fe43f596301e39fc0f6593` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:54.319986 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d879e3b0-2258-4cd2-a995-e29e196396ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9f3451911fe43f596301e39fc0f6593)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:54.322407 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d879e3b0-2258-4cd2-a995-e29e196396ab admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:54.344161 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ed61292d-a05a-4e87-9b12-06e910cd380a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b3d710f203542158ab1fbba3d65c720'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:54.365236 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ed61292d-a05a-4e87-9b12-06e910cd380a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b3d710f203542158ab1fbba3d65c720'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:54.366960 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ed61292d-a05a-4e87-9b12-06e910cd380a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b3d710f203542158ab1fbba3d65c720'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:54.403789 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ed61292d-a05a-4e87-9b12-06e910cd380a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b3d710f203542158ab1fbba3d65c720'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:54.419603 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ed61292d-a05a-4e87-9b12-06e910cd380a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b3d710f203542158ab1fbba3d65c720'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:54.432936 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 739/1488] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 03:33:54 2022] DELETE /identity/v3/users/8b3d710f203542158ab1fbba3d65c720 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:54.442605 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d879e3b0-2258-4cd2-a995-e29e196396ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9f3451911fe43f596301e39fc0f6593', 'request_id': 'req-d879e3b0-2258-4cd2-a995-e29e196396ab'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:54.452177 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fd12a4-0374-4dab-a38a-471095c4dceb None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:54.455571 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fd12a4-0374-4dab-a38a-471095c4dceb None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:54.456053 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fd12a4-0374-4dab-a38a-471095c4dceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:54.457565 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d879e3b0-2258-4cd2-a995-e29e196396ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9f3451911fe43f596301e39fc0f6593', 'request_id': 'req-d879e3b0-2258-4cd2-a995-e29e196396ab'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:54.466702 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d879e3b0-2258-4cd2-a995-e29e196396ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9f3451911fe43f596301e39fc0f6593', 'request_id': 'req-d879e3b0-2258-4cd2-a995-e29e196396ab'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:54.513852 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d879e3b0-2258-4cd2-a995-e29e196396ab admin admin] The token cache is being invalidate because project f9f3451911fe43f596301e39fc0f6593 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 03:33:54.515967 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d879e3b0-2258-4cd2-a995-e29e196396ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:54.516225 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-31fd12a4-0374-4dab-a38a-471095c4dceb None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:54.520375 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d879e3b0-2258-4cd2-a995-e29e196396ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:54.533243 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 750/1489] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 03:33:54 2022] DELETE /identity/v3/projects/f9f3451911fe43f596301e39fc0f6593 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 03:33:54.535072 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d879e3b0-2258-4cd2-a995-e29e196396ab admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:54.539092 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d879e3b0-2258-4cd2-a995-e29e196396ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:54.546912 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-31fd12a4-0374-4dab-a38a-471095c4dceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:54.676400 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 740/1490] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:54.677903 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:54.700012 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fd12a4-0374-4dab-a38a-471095c4dceb None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:54.701999 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-31fd12a4-0374-4dab-a38a-471095c4dceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:54.752498 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:54.764660 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d879e3b0-2258-4cd2-a995-e29e196396ab admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:54.792517 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bddd42-88ce-4eb4-b8f7-1454515da591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:54.795715 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bddd42-88ce-4eb4-b8f7-1454515da591 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:54.801025 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bddd42-88ce-4eb4-b8f7-1454515da591 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:54.801954 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bddd42-88ce-4eb4-b8f7-1454515da591 admin admin] PATH_INFO: `/v3/users/f15d53e7dec2486985d031184ba4f3fa` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:54.825947 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fd12a4-0374-4dab-a38a-471095c4dceb None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:54.828226 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bddd42-88ce-4eb4-b8f7-1454515da591 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f15d53e7dec2486985d031184ba4f3fa)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:54.830479 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad33a41-0d6b-438a-829d-512ed81037d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:54.831214 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad33a41-0d6b-438a-829d-512ed81037d1 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:54.831608 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad33a41-0d6b-438a-829d-512ed81037d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:54.831999 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad33a41-0d6b-438a-829d-512ed81037d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:54.841979 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04bddd42-88ce-4eb4-b8f7-1454515da591 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:54.844904 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad33a41-0d6b-438a-829d-512ed81037d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:54.846593 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad33a41-0d6b-438a-829d-512ed81037d1 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:54.858382 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 741/1491] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:33:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:33:54.912060 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d643755b-db5e-4382-b833-ff281b2899ae None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:54.912888 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d643755b-db5e-4382-b833-ff281b2899ae None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:54.913267 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d643755b-db5e-4382-b833-ff281b2899ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:54.976324 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d643755b-db5e-4382-b833-ff281b2899ae None None] MFA Rules not processed for user `f07e1b51b00f42b7b74c3689268635cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:54.979454 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d643755b-db5e-4382-b833-ff281b2899ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:54.990941 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 742/1492] 10.222.0.18 () {58 vars in 1015 bytes} [Wed Nov 2 03:33:54 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:55.002173 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-04bddd42-88ce-4eb4-b8f7-1454515da591 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f15d53e7dec2486985d031184ba4f3fa'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:55.003104 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-04bddd42-88ce-4eb4-b8f7-1454515da591 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f15d53e7dec2486985d031184ba4f3fa'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:55.003885 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-04bddd42-88ce-4eb4-b8f7-1454515da591 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f15d53e7dec2486985d031184ba4f3fa'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:55.023357 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-04bddd42-88ce-4eb4-b8f7-1454515da591 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f15d53e7dec2486985d031184ba4f3fa'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:55.043465 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-04bddd42-88ce-4eb4-b8f7-1454515da591 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f15d53e7dec2486985d031184ba4f3fa'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:55.083130 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 751/1493] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 03:33:54 2022] DELETE /identity/v3/users/f15d53e7dec2486985d031184ba4f3fa => generated 0 bytes in 548 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:55.096253 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d643755b-db5e-4382-b833-ff281b2899ae None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:55.097778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d643755b-db5e-4382-b833-ff281b2899ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:55.139544 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d643755b-db5e-4382-b833-ff281b2899ae None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:55.352739 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e9a6cd-35e0-4ec7-b71a-9a1ff171a752 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:55.353380 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e9a6cd-35e0-4ec7-b71a-9a1ff171a752 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:55.357479 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e9a6cd-35e0-4ec7-b71a-9a1ff171a752 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:55.357788 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e9a6cd-35e0-4ec7-b71a-9a1ff171a752 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:55.361543 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-35e9a6cd-35e0-4ec7-b71a-9a1ff171a752 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:55.385458 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e9a6cd-35e0-4ec7-b71a-9a1ff171a752 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:55.388891 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e9a6cd-35e0-4ec7-b71a-9a1ff171a752 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:55.416259 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 743/1494] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:33:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:33:56.495067 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bddd42-88ce-4eb4-b8f7-1454515da591 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:56.497140 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-04bddd42-88ce-4eb4-b8f7-1454515da591 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:56.927683 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bddd42-88ce-4eb4-b8f7-1454515da591 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:56.968380 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edaaa0d-cdca-46ed-8ab6-4f19b0482485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:56.969029 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edaaa0d-cdca-46ed-8ab6-4f19b0482485 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:56.969313 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edaaa0d-cdca-46ed-8ab6-4f19b0482485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:56.969656 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edaaa0d-cdca-46ed-8ab6-4f19b0482485 admin admin] PATH_INFO: `/v3/projects/ccc96c73927a495b9e183382a7252fa5` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:56.988663 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edaaa0d-cdca-46ed-8ab6-4f19b0482485 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccc96c73927a495b9e183382a7252fa5)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:56.993307 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edaaa0d-cdca-46ed-8ab6-4f19b0482485 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:57.239150 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:57.243849 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2edaaa0d-cdca-46ed-8ab6-4f19b0482485 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccc96c73927a495b9e183382a7252fa5', 'request_id': 'req-2edaaa0d-cdca-46ed-8ab6-4f19b0482485'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:57.261739 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2edaaa0d-cdca-46ed-8ab6-4f19b0482485 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccc96c73927a495b9e183382a7252fa5', 'request_id': 'req-2edaaa0d-cdca-46ed-8ab6-4f19b0482485'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:57.275790 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2edaaa0d-cdca-46ed-8ab6-4f19b0482485 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccc96c73927a495b9e183382a7252fa5', 'request_id': 'req-2edaaa0d-cdca-46ed-8ab6-4f19b0482485'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:57.306647 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2edaaa0d-cdca-46ed-8ab6-4f19b0482485 admin admin] The token cache is being invalidate because project ccc96c73927a495b9e183382a7252fa5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 03:33:57.308324 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2edaaa0d-cdca-46ed-8ab6-4f19b0482485 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:57.310716 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2edaaa0d-cdca-46ed-8ab6-4f19b0482485 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:57.318397 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 752/1495] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 03:33:56 2022] DELETE /identity/v3/projects/ccc96c73927a495b9e183382a7252fa5 => generated 0 bytes in 824 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:57.322125 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e9a6cd-35e0-4ec7-b71a-9a1ff171a752 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:57.341591 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e9a6cd-35e0-4ec7-b71a-9a1ff171a752 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:57.506526 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edaaa0d-cdca-46ed-8ab6-4f19b0482485 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:57.508581 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-2edaaa0d-cdca-46ed-8ab6-4f19b0482485 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:57.705003 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2959e986-efbb-4cc7-ad10-995707153730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:57.705884 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959e986-efbb-4cc7-ad10-995707153730 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:57.709172 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959e986-efbb-4cc7-ad10-995707153730 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:57.709172 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2959e986-efbb-4cc7-ad10-995707153730 admin admin] PATH_INFO: `/v3/users/170d9d09007d4feeadcb18b0befc1165` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:57.713422 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2959e986-efbb-4cc7-ad10-995707153730 admin admin] RBAC: Authorizing `identity:delete_user(user_id=170d9d09007d4feeadcb18b0befc1165)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:57.715985 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2959e986-efbb-4cc7-ad10-995707153730 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:57.735854 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edaaa0d-cdca-46ed-8ab6-4f19b0482485 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:57.748656 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566224cf-1efa-4a88-b215-c22711555a6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:57.749273 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566224cf-1efa-4a88-b215-c22711555a6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:57.749589 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566224cf-1efa-4a88-b215-c22711555a6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:57.749858 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566224cf-1efa-4a88-b215-c22711555a6a admin admin] PATH_INFO: `/v3/projects/64dfeff7c30e49f68600f1f490976e85` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:57.753077 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566224cf-1efa-4a88-b215-c22711555a6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=64dfeff7c30e49f68600f1f490976e85)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:57.754998 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566224cf-1efa-4a88-b215-c22711555a6a admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:57.812699 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2959e986-efbb-4cc7-ad10-995707153730 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '170d9d09007d4feeadcb18b0befc1165'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:57.814323 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:57.831696 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2959e986-efbb-4cc7-ad10-995707153730 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '170d9d09007d4feeadcb18b0befc1165'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:57.832932 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2959e986-efbb-4cc7-ad10-995707153730 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '170d9d09007d4feeadcb18b0befc1165'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:57.864750 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2959e986-efbb-4cc7-ad10-995707153730 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '170d9d09007d4feeadcb18b0befc1165'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:57.876393 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-566224cf-1efa-4a88-b215-c22711555a6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64dfeff7c30e49f68600f1f490976e85', 'request_id': 'req-566224cf-1efa-4a88-b215-c22711555a6a'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:57.880603 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2959e986-efbb-4cc7-ad10-995707153730 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '170d9d09007d4feeadcb18b0befc1165'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:57.886607 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-566224cf-1efa-4a88-b215-c22711555a6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64dfeff7c30e49f68600f1f490976e85', 'request_id': 'req-566224cf-1efa-4a88-b215-c22711555a6a'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:57.890012 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 744/1496] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 03:33:57 2022] DELETE /identity/v3/users/170d9d09007d4feeadcb18b0befc1165 => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:57.900996 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-566224cf-1efa-4a88-b215-c22711555a6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64dfeff7c30e49f68600f1f490976e85', 'request_id': 'req-566224cf-1efa-4a88-b215-c22711555a6a'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:57.919143 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-566224cf-1efa-4a88-b215-c22711555a6a admin admin] The token cache is being invalidate because project 64dfeff7c30e49f68600f1f490976e85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 03:33:57.919649 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-566224cf-1efa-4a88-b215-c22711555a6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:57.921056 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-566224cf-1efa-4a88-b215-c22711555a6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:57.924233 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 753/1497] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 03:33:57 2022] DELETE /identity/v3/projects/64dfeff7c30e49f68600f1f490976e85 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:57.929336 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c70d1-4495-439c-b4d9-b51f82aacc93 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:57.929776 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c70d1-4495-439c-b4d9-b51f82aacc93 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:57.930402 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7c70d1-4495-439c-b4d9-b51f82aacc93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:57.967262 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-2d7c70d1-4495-439c-b4d9-b51f82aacc93 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:57.968982 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2d7c70d1-4495-439c-b4d9-b51f82aacc93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:58.062180 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 745/1498] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:58.064536 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d38b98-a149-44f6-875e-f96fe883b02b None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:58.064830 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d38b98-a149-44f6-875e-f96fe883b02b None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:58.065110 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d38b98-a149-44f6-875e-f96fe883b02b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:58.104019 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-c7d38b98-a149-44f6-875e-f96fe883b02b None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:58.105283 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-c7d38b98-a149-44f6-875e-f96fe883b02b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:58.133616 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 754/1499] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:58.137939 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7c70d1-4495-439c-b4d9-b51f82aacc93 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:58.150416 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7c70d1-4495-439c-b4d9-b51f82aacc93 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:58.157935 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2c2386-25bf-4ee8-b66f-69840d2b5b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:58.158567 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c2386-25bf-4ee8-b66f-69840d2b5b4a admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:58.158843 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c2386-25bf-4ee8-b66f-69840d2b5b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:58.159120 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2c2386-25bf-4ee8-b66f-69840d2b5b4a admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:58.160037 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2c2386-25bf-4ee8-b66f-69840d2b5b4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:33:58.160149 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2c2386-25bf-4ee8-b66f-69840d2b5b4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:58.164646 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2c2386-25bf-4ee8-b66f-69840d2b5b4a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:58.194269 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 746/1500] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 03:33:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:33:58.195811 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d38b98-a149-44f6-875e-f96fe883b02b None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:58.206966 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d38b98-a149-44f6-875e-f96fe883b02b None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:58.224812 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d846ee1-8834-4c41-a87e-89d24f4d8324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:58.225466 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d846ee1-8834-4c41-a87e-89d24f4d8324 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:58.225742 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d846ee1-8834-4c41-a87e-89d24f4d8324 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:58.226013 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d846ee1-8834-4c41-a87e-89d24f4d8324 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:58.226685 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d846ee1-8834-4c41-a87e-89d24f4d8324 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:58.230036 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d846ee1-8834-4c41-a87e-89d24f4d8324 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:58.267012 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 755/1501] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:33:58 2022] POST /identity/v3/projects => generated 383 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:33:58.271189 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2c2386-25bf-4ee8-b66f-69840d2b5b4a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:58.287628 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2c2386-25bf-4ee8-b66f-69840d2b5b4a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:58.408482 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c1c030-ded9-4e48-b93b-89dff3e0f349 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:58.409154 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1c030-ded9-4e48-b93b-89dff3e0f349 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:58.409425 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1c030-ded9-4e48-b93b-89dff3e0f349 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:58.409698 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1c030-ded9-4e48-b93b-89dff3e0f349 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:58.410398 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c1c030-ded9-4e48-b93b-89dff3e0f349 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:58.412487 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c1c030-ded9-4e48-b93b-89dff3e0f349 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:58.446107 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 747/1502] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:33:58 2022] POST /identity/v3/users => generated 292 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:33:58.450449 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d846ee1-8834-4c41-a87e-89d24f4d8324 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:58.462499 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d846ee1-8834-4c41-a87e-89d24f4d8324 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:58.467941 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6e21f1-7613-4fd6-a91d-0ad6f8a9185c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:58.468700 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6e21f1-7613-4fd6-a91d-0ad6f8a9185c admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:58.469918 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6e21f1-7613-4fd6-a91d-0ad6f8a9185c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:58.472744 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6e21f1-7613-4fd6-a91d-0ad6f8a9185c admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:58.473644 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6e21f1-7613-4fd6-a91d-0ad6f8a9185c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:58.475353 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6e21f1-7613-4fd6-a91d-0ad6f8a9185c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:58.507495 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 756/1503] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:33:58.508398 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c1c030-ded9-4e48-b93b-89dff3e0f349 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:58.522103 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c1c030-ded9-4e48-b93b-89dff3e0f349 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:58.555685 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7025a40c-b4e5-4960-a326-b97bd64ed24c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:58.556355 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7025a40c-b4e5-4960-a326-b97bd64ed24c admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:58.556635 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7025a40c-b4e5-4960-a326-b97bd64ed24c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:58.556908 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7025a40c-b4e5-4960-a326-b97bd64ed24c admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:58.557557 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7025a40c-b4e5-4960-a326-b97bd64ed24c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:58.559155 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7025a40c-b4e5-4960-a326-b97bd64ed24c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:58.584956 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 748/1504] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:33:58.589273 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6e21f1-7613-4fd6-a91d-0ad6f8a9185c admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:58.611965 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6e21f1-7613-4fd6-a91d-0ad6f8a9185c admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:58.620693 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4010d664-fef7-40ee-b5b1-1cbb26b19ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:58.621395 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4010d664-fef7-40ee-b5b1-1cbb26b19ae3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:58.621678 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4010d664-fef7-40ee-b5b1-1cbb26b19ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:58.621957 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4010d664-fef7-40ee-b5b1-1cbb26b19ae3 admin admin] PATH_INFO: `/v3/projects/f15f0c355f4448d99f063cb8edcb3d79/users/bbb6538a3e694bbabc18afa9db663e45/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:58.656697 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4010d664-fef7-40ee-b5b1-1cbb26b19ae3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f15f0c355f4448d99f063cb8edcb3d79, user_id=bbb6538a3e694bbabc18afa9db663e45, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:58.659989 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4010d664-fef7-40ee-b5b1-1cbb26b19ae3 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:58.680549 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:58.700980 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39295261-4244-4ce4-a830-1113ccb145a3 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:58.701265 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39295261-4244-4ce4-a830-1113ccb145a3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:58.701542 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39295261-4244-4ce4-a830-1113ccb145a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:58.705945 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 757/1505] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:33:58 2022] PUT /identity/v3/projects/f15f0c355f4448d99f063cb8edcb3d79/users/bbb6538a3e694bbabc18afa9db663e45/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:58.781186 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-39295261-4244-4ce4-a830-1113ccb145a3 None None] MFA Rules not processed for user `bbb6538a3e694bbabc18afa9db663e45`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:58.785747 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-39295261-4244-4ce4-a830-1113ccb145a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:58.822291 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:58.893585 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 749/1506] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:58.925249 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4010d664-fef7-40ee-b5b1-1cbb26b19ae3 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:58.927574 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-4010d664-fef7-40ee-b5b1-1cbb26b19ae3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:59.032427 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4010d664-fef7-40ee-b5b1-1cbb26b19ae3 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:59.042357 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460f69b6-9719-4e06-8e85-fcb1103ffc3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:59.043361 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460f69b6-9719-4e06-8e85-fcb1103ffc3d service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:59.043784 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460f69b6-9719-4e06-8e85-fcb1103ffc3d service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:59.044236 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460f69b6-9719-4e06-8e85-fcb1103ffc3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:59.062767 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460f69b6-9719-4e06-8e85-fcb1103ffc3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:59.065123 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460f69b6-9719-4e06-8e85-fcb1103ffc3d service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:59.080842 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 758/1507] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:33:58 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:33:59.114393 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39295261-4244-4ce4-a830-1113ccb145a3 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:59.238030 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39295261-4244-4ce4-a830-1113ccb145a3 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:59.243861 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c949f0-feaa-4d06-87b3-b71c7e5962d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:59.244863 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c949f0-feaa-4d06-87b3-b71c7e5962d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:59.245317 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c949f0-feaa-4d06-87b3-b71c7e5962d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:59.246132 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c949f0-feaa-4d06-87b3-b71c7e5962d2 admin admin] PATH_INFO: `/v3/users/bbb6538a3e694bbabc18afa9db663e45` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:59.247830 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c949f0-feaa-4d06-87b3-b71c7e5962d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbb6538a3e694bbabc18afa9db663e45)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:59.250084 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c949f0-feaa-4d06-87b3-b71c7e5962d2 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:59.326063 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-c2c949f0-feaa-4d06-87b3-b71c7e5962d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbb6538a3e694bbabc18afa9db663e45'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:59.334404 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-c2c949f0-feaa-4d06-87b3-b71c7e5962d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbb6538a3e694bbabc18afa9db663e45'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:59.335353 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-c2c949f0-feaa-4d06-87b3-b71c7e5962d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbb6538a3e694bbabc18afa9db663e45'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:59.353750 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-c2c949f0-feaa-4d06-87b3-b71c7e5962d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbb6538a3e694bbabc18afa9db663e45'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:59.369901 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-c2c949f0-feaa-4d06-87b3-b71c7e5962d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbb6538a3e694bbabc18afa9db663e45'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:59.381456 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 750/1508] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 03:33:59 2022] DELETE /identity/v3/users/bbb6538a3e694bbabc18afa9db663e45 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:59.385728 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1273b9-73f3-48a4-a0a2-47748c3a02db None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:59.386087 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1273b9-73f3-48a4-a0a2-47748c3a02db None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:59.386351 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1273b9-73f3-48a4-a0a2-47748c3a02db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:59.409726 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-3f1273b9-73f3-48a4-a0a2-47748c3a02db None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:59.411163 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-3f1273b9-73f3-48a4-a0a2-47748c3a02db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:59.473119 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c949f0-feaa-4d06-87b3-b71c7e5962d2 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:59.474701 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c2c949f0-feaa-4d06-87b3-b71c7e5962d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:59.501678 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 759/1509] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:59.547110 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1273b9-73f3-48a4-a0a2-47748c3a02db None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:59.548830 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-3f1273b9-73f3-48a4-a0a2-47748c3a02db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:59.633427 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:59.687392 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c949f0-feaa-4d06-87b3-b71c7e5962d2 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:59.697783 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75db6717-b2d9-49d4-a11e-be880650b47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:59.698657 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75db6717-b2d9-49d4-a11e-be880650b47d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:59.698940 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75db6717-b2d9-49d4-a11e-be880650b47d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:59.699222 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75db6717-b2d9-49d4-a11e-be880650b47d admin admin] PATH_INFO: `/v3/projects/5cddc0bafebf45db8e4bb064deae81bd` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:59.700748 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75db6717-b2d9-49d4-a11e-be880650b47d admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cddc0bafebf45db8e4bb064deae81bd)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:59.703373 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75db6717-b2d9-49d4-a11e-be880650b47d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:59.704347 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1273b9-73f3-48a4-a0a2-47748c3a02db None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:59.709086 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48953dae-0e03-4358-83df-16d57efb1229 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:59.710611 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48953dae-0e03-4358-83df-16d57efb1229 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:59.710895 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48953dae-0e03-4358-83df-16d57efb1229 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:59.711335 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48953dae-0e03-4358-83df-16d57efb1229 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:59.724604 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48953dae-0e03-4358-83df-16d57efb1229 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:59.727441 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48953dae-0e03-4358-83df-16d57efb1229 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:59.748779 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 760/1510] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:33:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 03:33:59.750425 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b95ee6-4d98-4724-993c-8dbbb976dd27 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:59.750834 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b95ee6-4d98-4724-993c-8dbbb976dd27 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:59.751195 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b95ee6-4d98-4724-993c-8dbbb976dd27 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:59.752536 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 761/1511] 10.222.0.26 () {54 vars in 921 bytes} [Wed Nov 2 03:33:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:33:59.758942 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfefa74f-3ebf-4bbe-9280-f867d9890300 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:59.759294 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfefa74f-3ebf-4bbe-9280-f867d9890300 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:59.759774 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfefa74f-3ebf-4bbe-9280-f867d9890300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:59.779915 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-75db6717-b2d9-49d4-a11e-be880650b47d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cddc0bafebf45db8e4bb064deae81bd', 'request_id': 'req-75db6717-b2d9-49d4-a11e-be880650b47d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:59.787697 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-75db6717-b2d9-49d4-a11e-be880650b47d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cddc0bafebf45db8e4bb064deae81bd', 'request_id': 'req-75db6717-b2d9-49d4-a11e-be880650b47d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:59.789403 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-bfefa74f-3ebf-4bbe-9280-f867d9890300 None None] MFA Rules not processed for user `44961b06151c4da382fcd54dd1f1e146`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:59.791457 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-bfefa74f-3ebf-4bbe-9280-f867d9890300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:59.796527 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-75db6717-b2d9-49d4-a11e-be880650b47d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cddc0bafebf45db8e4bb064deae81bd', 'request_id': 'req-75db6717-b2d9-49d4-a11e-be880650b47d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:59.809116 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-75db6717-b2d9-49d4-a11e-be880650b47d admin admin] The token cache is being invalidate because project 5cddc0bafebf45db8e4bb064deae81bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 03:33:59.809409 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-75db6717-b2d9-49d4-a11e-be880650b47d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:59.810051 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-75db6717-b2d9-49d4-a11e-be880650b47d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:33:59.812344 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 751/1512] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 03:33:59 2022] DELETE /identity/v3/projects/5cddc0bafebf45db8e4bb064deae81bd => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:33:59.818012 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bbc16d-9f90-421b-9c8b-f0c743236340 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:59.818418 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bbc16d-9f90-421b-9c8b-f0c743236340 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:59.818782 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bbc16d-9f90-421b-9c8b-f0c743236340 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:59.841002 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:33:59.844203 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-52bbc16d-9f90-421b-9c8b-f0c743236340 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:59.845925 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-52bbc16d-9f90-421b-9c8b-f0c743236340 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:59.876113 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 762/1513] 10.222.0.26 () {58 vars in 1018 bytes} [Wed Nov 2 03:33:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 03:33:59.877843 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05953391-0bdb-4b60-be4f-c331308254ef None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:59.878381 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05953391-0bdb-4b60-be4f-c331308254ef None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:59.879555 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05953391-0bdb-4b60-be4f-c331308254ef None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:59.880940 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 763/1514] 10.222.0.26 () {54 vars in 921 bytes} [Wed Nov 2 03:33:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:33:59.890232 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05953391-0bdb-4b60-be4f-c331308254ef None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:33:59.892982 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-05953391-0bdb-4b60-be4f-c331308254ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:59.915453 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05953391-0bdb-4b60-be4f-c331308254ef None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:33:59.922433 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc73f75-2257-4759-a157-c4c8a8c3b7fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:33:59.923935 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc73f75-2257-4759-a157-c4c8a8c3b7fa service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:59.924358 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc73f75-2257-4759-a157-c4c8a8c3b7fa service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:59.925132 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc73f75-2257-4759-a157-c4c8a8c3b7fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:59.928734 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 752/1515] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 03:33:59.930026 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef665def-3d13-48dc-bc7c-3924f7dd2c0d None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:59.930612 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef665def-3d13-48dc-bc7c-3924f7dd2c0d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:59.930612 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef665def-3d13-48dc-bc7c-3924f7dd2c0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:33:59.948658 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc73f75-2257-4759-a157-c4c8a8c3b7fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:33:59.950904 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc73f75-2257-4759-a157-c4c8a8c3b7fa service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:33:59.964484 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-ef665def-3d13-48dc-bc7c-3924f7dd2c0d None None] MFA Rules not processed for user `44961b06151c4da382fcd54dd1f1e146`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:33:59.966650 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ef665def-3d13-48dc-bc7c-3924f7dd2c0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:33:59.968068 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 764/1516] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:33:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:33:59.980840 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 753/1517] 10.222.0.26 () {58 vars in 1018 bytes} [Wed Nov 2 03:33:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:33:59.982785 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae44f09-d75e-4566-a1ea-5103dc98bfe2 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:33:59.984257 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae44f09-d75e-4566-a1ea-5103dc98bfe2 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:33:59.984603 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae44f09-d75e-4566-a1ea-5103dc98bfe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:00.026258 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-aae44f09-d75e-4566-a1ea-5103dc98bfe2 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:34:00.028113 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-aae44f09-d75e-4566-a1ea-5103dc98bfe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:00.040852 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 765/1518] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:33:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:34:00.041988 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef665def-3d13-48dc-bc7c-3924f7dd2c0d None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:00.056941 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef665def-3d13-48dc-bc7c-3924f7dd2c0d None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:00.065574 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff617c1-49c6-4588-8519-7cbf8f6e1412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:00.066257 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff617c1-49c6-4588-8519-7cbf8f6e1412 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:00.066479 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff617c1-49c6-4588-8519-7cbf8f6e1412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:00.067396 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff617c1-49c6-4588-8519-7cbf8f6e1412 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:00.068094 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff617c1-49c6-4588-8519-7cbf8f6e1412 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:34:00.068399 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff617c1-49c6-4588-8519-7cbf8f6e1412 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:00.070645 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff617c1-49c6-4588-8519-7cbf8f6e1412 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:00.098779 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 754/1519] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 03:34:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:34:00.103557 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae44f09-d75e-4566-a1ea-5103dc98bfe2 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:00.117269 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae44f09-d75e-4566-a1ea-5103dc98bfe2 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:00.123142 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09126283-b922-40fe-9e48-4162ff997918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:00.123880 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09126283-b922-40fe-9e48-4162ff997918 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:00.124265 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09126283-b922-40fe-9e48-4162ff997918 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:00.124655 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09126283-b922-40fe-9e48-4162ff997918 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:00.125479 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09126283-b922-40fe-9e48-4162ff997918 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:00.128077 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09126283-b922-40fe-9e48-4162ff997918 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:00.163429 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 766/1520] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:34:00 2022] POST /identity/v3/projects => generated 377 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:34:00.167192 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff617c1-49c6-4588-8519-7cbf8f6e1412 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:00.193752 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff617c1-49c6-4588-8519-7cbf8f6e1412 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:00.195407 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09126283-b922-40fe-9e48-4162ff997918 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:00.197106 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-09126283-b922-40fe-9e48-4162ff997918 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:00.267099 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:34:00.285760 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146a077c-e076-4e06-a2bc-3ec2030bbfbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:00.287279 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146a077c-e076-4e06-a2bc-3ec2030bbfbf admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:00.287677 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146a077c-e076-4e06-a2bc-3ec2030bbfbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:00.288494 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146a077c-e076-4e06-a2bc-3ec2030bbfbf admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:00.289487 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146a077c-e076-4e06-a2bc-3ec2030bbfbf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:00.292702 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146a077c-e076-4e06-a2bc-3ec2030bbfbf admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:00.322333 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09126283-b922-40fe-9e48-4162ff997918 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:00.330506 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a90a71-fb1a-41d5-af63-f2cd79db2919 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:00.331204 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a90a71-fb1a-41d5-af63-f2cd79db2919 service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:00.331510 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a90a71-fb1a-41d5-af63-f2cd79db2919 service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:00.331785 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a90a71-fb1a-41d5-af63-f2cd79db2919 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:00.332549 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 755/1521] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:34:00 2022] POST /identity/v3/users => generated 289 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:34:00.336252 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146a077c-e076-4e06-a2bc-3ec2030bbfbf admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:00.343815 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a90a71-fb1a-41d5-af63-f2cd79db2919 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:00.345992 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a90a71-fb1a-41d5-af63-f2cd79db2919 service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:00.349521 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146a077c-e076-4e06-a2bc-3ec2030bbfbf admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:00.357360 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0740ec45-4c2d-4d9d-9f47-1c077b18179b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:00.357999 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740ec45-4c2d-4d9d-9f47-1c077b18179b admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:00.358279 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740ec45-4c2d-4d9d-9f47-1c077b18179b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:00.358549 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0740ec45-4c2d-4d9d-9f47-1c077b18179b admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:00.359308 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 767/1522] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:34:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 03:34:00.368543 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0740ec45-4c2d-4d9d-9f47-1c077b18179b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:00.371196 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0740ec45-4c2d-4d9d-9f47-1c077b18179b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:00.390675 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 756/1523] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:34:00 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:34:00.391497 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a90a71-fb1a-41d5-af63-f2cd79db2919 service nova] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:00.403690 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a90a71-fb1a-41d5-af63-f2cd79db2919 service nova] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:00.411858 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0740ec45-4c2d-4d9d-9f47-1c077b18179b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:00.414501 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0740ec45-4c2d-4d9d-9f47-1c077b18179b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:00.416713 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50d5069-9dd4-48df-a25e-57a2ebb16062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:00.417853 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d5069-9dd4-48df-a25e-57a2ebb16062 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:00.418521 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d5069-9dd4-48df-a25e-57a2ebb16062 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:00.418890 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50d5069-9dd4-48df-a25e-57a2ebb16062 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:00.419757 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50d5069-9dd4-48df-a25e-57a2ebb16062 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:00.423033 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50d5069-9dd4-48df-a25e-57a2ebb16062 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:00.431289 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0740ec45-4c2d-4d9d-9f47-1c077b18179b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:00.442390 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 768/1524] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:34:00 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:34:00.443614 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50d5069-9dd4-48df-a25e-57a2ebb16062 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:00.448214 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c08d1f1-88a2-498b-88e3-1d8c7d32888c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:00.449515 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c08d1f1-88a2-498b-88e3-1d8c7d32888c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:00.449960 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c08d1f1-88a2-498b-88e3-1d8c7d32888c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:00.450369 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c08d1f1-88a2-498b-88e3-1d8c7d32888c admin admin] PATH_INFO: `/v3/projects/f15f0c355f4448d99f063cb8edcb3d79` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:00.451885 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c08d1f1-88a2-498b-88e3-1d8c7d32888c admin admin] RBAC: Authorizing `identity:delete_project(project_id=f15f0c355f4448d99f063cb8edcb3d79)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:00.454855 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c08d1f1-88a2-498b-88e3-1d8c7d32888c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:00.456278 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50d5069-9dd4-48df-a25e-57a2ebb16062 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:00.461184 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6da7cd1-6426-4014-b7cf-93d74dfdd095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:00.462263 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da7cd1-6426-4014-b7cf-93d74dfdd095 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:00.462550 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da7cd1-6426-4014-b7cf-93d74dfdd095 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:00.462895 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6da7cd1-6426-4014-b7cf-93d74dfdd095 admin admin] PATH_INFO: `/v3/projects/6c1510f29e8a4608992064d6d60f2951/users/149624e2c0584cf39b6cc468668bc4d1/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:00.491253 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6da7cd1-6426-4014-b7cf-93d74dfdd095 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c1510f29e8a4608992064d6d60f2951, user_id=149624e2c0584cf39b6cc468668bc4d1, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:00.494499 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6da7cd1-6426-4014-b7cf-93d74dfdd095 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:00.512394 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-3c08d1f1-88a2-498b-88e3-1d8c7d32888c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f15f0c355f4448d99f063cb8edcb3d79', 'request_id': 'req-3c08d1f1-88a2-498b-88e3-1d8c7d32888c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:00.515833 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 769/1525] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:34:00 2022] PUT /identity/v3/projects/6c1510f29e8a4608992064d6d60f2951/users/149624e2c0584cf39b6cc468668bc4d1/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:34:00.519235 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-3c08d1f1-88a2-498b-88e3-1d8c7d32888c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f15f0c355f4448d99f063cb8edcb3d79', 'request_id': 'req-3c08d1f1-88a2-498b-88e3-1d8c7d32888c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:00.528609 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-3c08d1f1-88a2-498b-88e3-1d8c7d32888c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f15f0c355f4448d99f063cb8edcb3d79', 'request_id': 'req-3c08d1f1-88a2-498b-88e3-1d8c7d32888c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:00.529751 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df3ef3-9890-48bc-bb39-b74f60696f6f None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:00.530558 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df3ef3-9890-48bc-bb39-b74f60696f6f None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:00.531012 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0df3ef3-9890-48bc-bb39-b74f60696f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:00.548363 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-3c08d1f1-88a2-498b-88e3-1d8c7d32888c admin admin] The token cache is being invalidate because project f15f0c355f4448d99f063cb8edcb3d79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 03:34:00.548951 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-3c08d1f1-88a2-498b-88e3-1d8c7d32888c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:00.549744 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-3c08d1f1-88a2-498b-88e3-1d8c7d32888c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:00.551587 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 757/1526] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 03:34:00 2022] DELETE /identity/v3/projects/f15f0c355f4448d99f063cb8edcb3d79 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:34:00.556687 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8660db5-8400-4321-bdbe-b18204b95a2f None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:00.557077 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8660db5-8400-4321-bdbe-b18204b95a2f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:00.557355 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8660db5-8400-4321-bdbe-b18204b95a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:00.564729 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-c0df3ef3-9890-48bc-bb39-b74f60696f6f None None] MFA Rules not processed for user `149624e2c0584cf39b6cc468668bc4d1`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:34:00.566011 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-c0df3ef3-9890-48bc-bb39-b74f60696f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:00.579172 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-e8660db5-8400-4321-bdbe-b18204b95a2f None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:34:00.580407 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e8660db5-8400-4321-bdbe-b18204b95a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:00.604371 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:34:00.651699 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 770/1527] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:34:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 03:34:00.667920 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0df3ef3-9890-48bc-bb39-b74f60696f6f None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:00.669508 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-c0df3ef3-9890-48bc-bb39-b74f60696f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:00.670896 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 758/1528] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:34:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 03:34:00.684506 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d41271-6524-4ad3-ba30-f9a674d95713 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:00.685230 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d41271-6524-4ad3-ba30-f9a674d95713 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:00.685730 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d41271-6524-4ad3-ba30-f9a674d95713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:00.710128 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-44d41271-6524-4ad3-ba30-f9a674d95713 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:34:00.711374 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-44d41271-6524-4ad3-ba30-f9a674d95713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:00.724242 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 759/1529] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:34:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:34:00.725253 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d41271-6524-4ad3-ba30-f9a674d95713 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:00.736952 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d41271-6524-4ad3-ba30-f9a674d95713 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:00.746668 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f1be93-0dbb-4fe9-b9df-0e4afa4fe775 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:00.747934 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f1be93-0dbb-4fe9-b9df-0e4afa4fe775 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:00.748428 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f1be93-0dbb-4fe9-b9df-0e4afa4fe775 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:00.748772 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f1be93-0dbb-4fe9-b9df-0e4afa4fe775 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:00.749658 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f1be93-0dbb-4fe9-b9df-0e4afa4fe775 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:34:00.750226 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f1be93-0dbb-4fe9-b9df-0e4afa4fe775 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:00.754955 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f1be93-0dbb-4fe9-b9df-0e4afa4fe775 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:00.768678 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 760/1530] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 03:34:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:34:00.783920 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f1be93-0dbb-4fe9-b9df-0e4afa4fe775 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:00.791811 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0df3ef3-9890-48bc-bb39-b74f60696f6f None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:00.794590 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f1be93-0dbb-4fe9-b9df-0e4afa4fe775 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:00.806703 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c60d3-f5df-4558-98fa-ee754addfa90 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50aab624ee9c4651a7cd42124bcc465f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:00.807443 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c60d3-f5df-4558-98fa-ee754addfa90 service glance] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:00.807790 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c60d3-f5df-4558-98fa-ee754addfa90 service glance] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:00.808153 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c60d3-f5df-4558-98fa-ee754addfa90 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:00.817757 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6759cd76-598c-442f-926b-d7f8274d6cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:00.818394 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6759cd76-598c-442f-926b-d7f8274d6cfc admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:00.818676 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6759cd76-598c-442f-926b-d7f8274d6cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:00.818951 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6759cd76-598c-442f-926b-d7f8274d6cfc admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:00.819597 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6759cd76-598c-442f-926b-d7f8274d6cfc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:00.821888 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6759cd76-598c-442f-926b-d7f8274d6cfc admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:00.824219 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521c60d3-f5df-4558-98fa-ee754addfa90 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:00.825751 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521c60d3-f5df-4558-98fa-ee754addfa90 service glance] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:00.840143 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 771/1531] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:34:00 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:34:00.866621 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c60d3-f5df-4558-98fa-ee754addfa90 service glance] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:00.872225 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 761/1532] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:34:00 2022] POST /identity/v3/projects => generated 375 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:34:00.894945 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c60d3-f5df-4558-98fa-ee754addfa90 service glance] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:00.961416 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2041a18b-d84b-4e04-b872-681a0ba0d412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:00.962034 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2041a18b-d84b-4e04-b872-681a0ba0d412 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:00.962327 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2041a18b-d84b-4e04-b872-681a0ba0d412 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:00.962734 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2041a18b-d84b-4e04-b872-681a0ba0d412 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:00.963265 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2041a18b-d84b-4e04-b872-681a0ba0d412 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:00.965240 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2041a18b-d84b-4e04-b872-681a0ba0d412 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:00.994350 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 772/1533] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:34:00 2022] POST /identity/v3/users => generated 288 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:34:00.995146 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6759cd76-598c-442f-926b-d7f8274d6cfc admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:01.007508 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6759cd76-598c-442f-926b-d7f8274d6cfc admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:01.012458 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a0832d-7186-43c7-985f-088259fb2121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:01.013076 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a0832d-7186-43c7-985f-088259fb2121 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:01.013463 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a0832d-7186-43c7-985f-088259fb2121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:01.013738 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a0832d-7186-43c7-985f-088259fb2121 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:01.014503 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a0832d-7186-43c7-985f-088259fb2121 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:01.016237 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a0832d-7186-43c7-985f-088259fb2121 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:01.022343 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:34:01.033278 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2041a18b-d84b-4e04-b872-681a0ba0d412 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:01.036159 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 762/1534] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:34:00 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:34:01.044110 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2041a18b-d84b-4e04-b872-681a0ba0d412 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:01.049497 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9a5bd7-23b5-4cc0-87b3-c9d40fe4a630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:01.050296 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a5bd7-23b5-4cc0-87b3-c9d40fe4a630 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:01.050690 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a5bd7-23b5-4cc0-87b3-c9d40fe4a630 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:01.051062 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9a5bd7-23b5-4cc0-87b3-c9d40fe4a630 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:01.051823 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9a5bd7-23b5-4cc0-87b3-c9d40fe4a630 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:01.053731 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9a5bd7-23b5-4cc0-87b3-c9d40fe4a630 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:01.071380 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 773/1535] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:34:01 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:34:01.075841 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a0832d-7186-43c7-985f-088259fb2121 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:01.087061 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a0832d-7186-43c7-985f-088259fb2121 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:01.104726 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b6bd2-1f85-42b9-9b17-ebac73121c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:01.105563 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b6bd2-1f85-42b9-9b17-ebac73121c90 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:01.105910 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b6bd2-1f85-42b9-9b17-ebac73121c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:01.106252 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734b6bd2-1f85-42b9-9b17-ebac73121c90 admin admin] PATH_INFO: `/v3/projects/40a4c1329f964f52b7a8003eebbc2692/users/65498b6991d946628ec9733660ee142c/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:01.128389 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734b6bd2-1f85-42b9-9b17-ebac73121c90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=40a4c1329f964f52b7a8003eebbc2692, user_id=65498b6991d946628ec9733660ee142c, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:01.131039 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734b6bd2-1f85-42b9-9b17-ebac73121c90 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:01.147104 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 763/1536] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:34:01 2022] PUT /identity/v3/projects/40a4c1329f964f52b7a8003eebbc2692/users/65498b6991d946628ec9733660ee142c/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 03:34:01.157575 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f3d1da-434d-40d4-9cae-e49d20d98c54 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:01.157863 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f3d1da-434d-40d4-9cae-e49d20d98c54 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:01.158138 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f3d1da-434d-40d4-9cae-e49d20d98c54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:01.200459 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-13f3d1da-434d-40d4-9cae-e49d20d98c54 None None] MFA Rules not processed for user `65498b6991d946628ec9733660ee142c`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:34:01.201686 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-13f3d1da-434d-40d4-9cae-e49d20d98c54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:01.242011 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:34:01.314571 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 774/1537] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:34:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:34:01.329301 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b6bd2-1f85-42b9-9b17-ebac73121c90 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:01.332271 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-734b6bd2-1f85-42b9-9b17-ebac73121c90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:01.454928 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734b6bd2-1f85-42b9-9b17-ebac73121c90 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:01.476041 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddb6130-d549-4488-a7ea-c8a99a562296 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:01.476041 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb6130-d549-4488-a7ea-c8a99a562296 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:01.476041 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb6130-d549-4488-a7ea-c8a99a562296 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:01.476041 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb6130-d549-4488-a7ea-c8a99a562296 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:01.498346 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddb6130-d549-4488-a7ea-c8a99a562296 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:01.500324 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddb6130-d549-4488-a7ea-c8a99a562296 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:01.518538 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 764/1538] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:34:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:34:02.237211 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a7436f-f784-46d7-9f2a-748a02c8e2e9 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:02.237751 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a7436f-f784-46d7-9f2a-748a02c8e2e9 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:02.237751 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a7436f-f784-46d7-9f2a-748a02c8e2e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:02.268883 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-d3a7436f-f784-46d7-9f2a-748a02c8e2e9 None None] MFA Rules not processed for user `da7ee607b66e43f3823d9d2faf373974`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:34:02.274177 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d3a7436f-f784-46d7-9f2a-748a02c8e2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:02.352176 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 775/1539] 10.222.0.26 () {58 vars in 1009 bytes} [Wed Nov 2 03:34:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:34:02.368747 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddb6130-d549-4488-a7ea-c8a99a562296 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:02.370314 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8ddb6130-d549-4488-a7ea-c8a99a562296 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:02.454487 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddb6130-d549-4488-a7ea-c8a99a562296 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:02.460252 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf09cd9-f503-4df1-a7c8-f486b740c467 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5193c40a917b4105ab8b27aead1685f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:02.460949 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf09cd9-f503-4df1-a7c8-f486b740c467 service swift] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:02.461334 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf09cd9-f503-4df1-a7c8-f486b740c467 service swift] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:02.461664 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf09cd9-f503-4df1-a7c8-f486b740c467 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:02.471921 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf09cd9-f503-4df1-a7c8-f486b740c467 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:02.473836 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf09cd9-f503-4df1-a7c8-f486b740c467 service swift] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:02.490896 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 765/1540] 10.222.0.26 () {60 vars in 1384 bytes} [Wed Nov 2 03:34:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 03:34:07.362932 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a7436f-f784-46d7-9f2a-748a02c8e2e9 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:07.443901 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a7436f-f784-46d7-9f2a-748a02c8e2e9 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:07.449656 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1834e6bf-b372-4f38-b297-005ece9d51c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:07.450506 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1834e6bf-b372-4f38-b297-005ece9d51c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:07.450961 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1834e6bf-b372-4f38-b297-005ece9d51c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:07.451219 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1834e6bf-b372-4f38-b297-005ece9d51c6 admin admin] PATH_INFO: `/v3/users/65498b6991d946628ec9733660ee142c` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:07.452420 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1834e6bf-b372-4f38-b297-005ece9d51c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65498b6991d946628ec9733660ee142c)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:07.455295 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1834e6bf-b372-4f38-b297-005ece9d51c6 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:07.559215 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1834e6bf-b372-4f38-b297-005ece9d51c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65498b6991d946628ec9733660ee142c'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:07.561205 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1834e6bf-b372-4f38-b297-005ece9d51c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65498b6991d946628ec9733660ee142c'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:07.564648 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1834e6bf-b372-4f38-b297-005ece9d51c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65498b6991d946628ec9733660ee142c'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:07.580442 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1834e6bf-b372-4f38-b297-005ece9d51c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65498b6991d946628ec9733660ee142c'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:07.589377 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1834e6bf-b372-4f38-b297-005ece9d51c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65498b6991d946628ec9733660ee142c'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:07.591908 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:34:07.612326 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 776/1541] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 03:34:07 2022] DELETE /identity/v3/users/65498b6991d946628ec9733660ee142c => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:34:07.616882 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497ac87a-8c2e-4f7d-84f0-2e826e6739af None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:07.617360 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497ac87a-8c2e-4f7d-84f0-2e826e6739af None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:07.617440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497ac87a-8c2e-4f7d-84f0-2e826e6739af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:07.654179 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-497ac87a-8c2e-4f7d-84f0-2e826e6739af None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:34:07.656212 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-497ac87a-8c2e-4f7d-84f0-2e826e6739af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:07.761064 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 766/1542] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:34:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 03:34:07.848477 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1834e6bf-b372-4f38-b297-005ece9d51c6 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:07.850055 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-1834e6bf-b372-4f38-b297-005ece9d51c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:08.028777 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1834e6bf-b372-4f38-b297-005ece9d51c6 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:08.037154 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbfc29e-e33e-4743-9000-28dc491fb31d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:08.041644 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbfc29e-e33e-4743-9000-28dc491fb31d service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:08.041962 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbfc29e-e33e-4743-9000-28dc491fb31d service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:08.042833 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbfc29e-e33e-4743-9000-28dc491fb31d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:08.074165 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbfc29e-e33e-4743-9000-28dc491fb31d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:08.076487 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbfc29e-e33e-4743-9000-28dc491fb31d service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:08.105462 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 777/1543] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:34:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:34:10.114773 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497ac87a-8c2e-4f7d-84f0-2e826e6739af None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:10.117028 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-497ac87a-8c2e-4f7d-84f0-2e826e6739af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:10.132307 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497ac87a-8c2e-4f7d-84f0-2e826e6739af None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:10.139269 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495b5235-f59f-47f3-944b-1e4c75c0dc25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:10.140182 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495b5235-f59f-47f3-944b-1e4c75c0dc25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:10.140640 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495b5235-f59f-47f3-944b-1e4c75c0dc25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:10.141167 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495b5235-f59f-47f3-944b-1e4c75c0dc25 admin admin] PATH_INFO: `/v3/projects/40a4c1329f964f52b7a8003eebbc2692` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:10.142642 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495b5235-f59f-47f3-944b-1e4c75c0dc25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40a4c1329f964f52b7a8003eebbc2692)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:10.146745 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495b5235-f59f-47f3-944b-1e4c75c0dc25 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:10.151302 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:34:10.242623 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-495b5235-f59f-47f3-944b-1e4c75c0dc25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40a4c1329f964f52b7a8003eebbc2692', 'request_id': 'req-495b5235-f59f-47f3-944b-1e4c75c0dc25'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:10.251062 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-495b5235-f59f-47f3-944b-1e4c75c0dc25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40a4c1329f964f52b7a8003eebbc2692', 'request_id': 'req-495b5235-f59f-47f3-944b-1e4c75c0dc25'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:10.262706 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-495b5235-f59f-47f3-944b-1e4c75c0dc25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40a4c1329f964f52b7a8003eebbc2692', 'request_id': 'req-495b5235-f59f-47f3-944b-1e4c75c0dc25'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:10.299915 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-495b5235-f59f-47f3-944b-1e4c75c0dc25 admin admin] The token cache is being invalidate because project 40a4c1329f964f52b7a8003eebbc2692 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 03:34:10.300233 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-495b5235-f59f-47f3-944b-1e4c75c0dc25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:10.304382 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-495b5235-f59f-47f3-944b-1e4c75c0dc25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:10.306027 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 767/1544] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 03:34:10 2022] DELETE /identity/v3/projects/40a4c1329f964f52b7a8003eebbc2692 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:34:10.310957 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5337be1c-0612-44bf-9c55-f423e9495942 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:10.311885 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5337be1c-0612-44bf-9c55-f423e9495942 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:10.312217 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5337be1c-0612-44bf-9c55-f423e9495942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:10.347659 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-5337be1c-0612-44bf-9c55-f423e9495942 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:34:10.350239 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-5337be1c-0612-44bf-9c55-f423e9495942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:10.450164 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 778/1545] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:34:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:34:10.450489 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0508d17-edf0-442f-b5d6-5f9edd8a8c95 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:10.451262 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0508d17-edf0-442f-b5d6-5f9edd8a8c95 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:10.451968 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0508d17-edf0-442f-b5d6-5f9edd8a8c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:10.496078 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-b0508d17-edf0-442f-b5d6-5f9edd8a8c95 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:34:10.499244 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b0508d17-edf0-442f-b5d6-5f9edd8a8c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:10.507526 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 768/1546] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:34:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:34:10.512015 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5337be1c-0612-44bf-9c55-f423e9495942 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:10.526261 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5337be1c-0612-44bf-9c55-f423e9495942 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:10.536028 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6333fe14-0521-4add-bc48-b4fc56edd3f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:10.536028 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6333fe14-0521-4add-bc48-b4fc56edd3f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:10.536028 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6333fe14-0521-4add-bc48-b4fc56edd3f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:10.536028 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6333fe14-0521-4add-bc48-b4fc56edd3f6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:10.536436 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6333fe14-0521-4add-bc48-b4fc56edd3f6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:34:10.536436 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6333fe14-0521-4add-bc48-b4fc56edd3f6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:10.540100 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6333fe14-0521-4add-bc48-b4fc56edd3f6 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:10.557776 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 779/1547] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 03:34:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:34:10.561250 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0508d17-edf0-442f-b5d6-5f9edd8a8c95 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:10.571598 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0508d17-edf0-442f-b5d6-5f9edd8a8c95 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:10.576882 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c023da-2ef0-46a5-a59c-215175a7b720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:10.577521 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c023da-2ef0-46a5-a59c-215175a7b720 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:10.577802 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c023da-2ef0-46a5-a59c-215175a7b720 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:10.578076 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c023da-2ef0-46a5-a59c-215175a7b720 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:10.578764 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c023da-2ef0-46a5-a59c-215175a7b720 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:10.580742 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c023da-2ef0-46a5-a59c-215175a7b720 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:10.619871 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 769/1548] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:34:10 2022] POST /identity/v3/projects => generated 369 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:34:10.621042 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6333fe14-0521-4add-bc48-b4fc56edd3f6 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:10.635838 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6333fe14-0521-4add-bc48-b4fc56edd3f6 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:10.745117 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e62b3-84fa-449f-a63f-208a55034b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:10.745917 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e62b3-84fa-449f-a63f-208a55034b5d admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:10.746264 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e62b3-84fa-449f-a63f-208a55034b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:10.746614 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91e62b3-84fa-449f-a63f-208a55034b5d admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:10.747924 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91e62b3-84fa-449f-a63f-208a55034b5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:10.750191 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91e62b3-84fa-449f-a63f-208a55034b5d admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:10.790156 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 780/1549] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:34:10 2022] POST /identity/v3/users => generated 285 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 03:34:10.795399 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c023da-2ef0-46a5-a59c-215175a7b720 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:10.816158 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c023da-2ef0-46a5-a59c-215175a7b720 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:10.821953 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d72d508-19a2-4683-9c18-a9c2e3c0e0d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:10.823438 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d72d508-19a2-4683-9c18-a9c2e3c0e0d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:10.824366 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d72d508-19a2-4683-9c18-a9c2e3c0e0d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:10.824366 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d72d508-19a2-4683-9c18-a9c2e3c0e0d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:10.824834 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d72d508-19a2-4683-9c18-a9c2e3c0e0d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:10.826572 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d72d508-19a2-4683-9c18-a9c2e3c0e0d5 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:10.847673 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 770/1550] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:34:10 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:34:10.851808 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e62b3-84fa-449f-a63f-208a55034b5d admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:10.866897 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91e62b3-84fa-449f-a63f-208a55034b5d admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:10.875171 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a42e382-bf0c-428f-9e18-05ea8d7b9465 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:10.876022 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42e382-bf0c-428f-9e18-05ea8d7b9465 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:10.877261 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42e382-bf0c-428f-9e18-05ea8d7b9465 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:10.877563 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a42e382-bf0c-428f-9e18-05ea8d7b9465 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:10.878236 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a42e382-bf0c-428f-9e18-05ea8d7b9465 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:10.881728 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a42e382-bf0c-428f-9e18-05ea8d7b9465 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:10.894992 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:34:10.907792 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 781/1551] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:34:10 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:34:10.912811 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d72d508-19a2-4683-9c18-a9c2e3c0e0d5 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:10.943661 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d72d508-19a2-4683-9c18-a9c2e3c0e0d5 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:10.956695 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b767e11-fe65-4efd-bd8d-4ae6a7cfa223 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:10.959285 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b767e11-fe65-4efd-bd8d-4ae6a7cfa223 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:10.959440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b767e11-fe65-4efd-bd8d-4ae6a7cfa223 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:10.959681 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b767e11-fe65-4efd-bd8d-4ae6a7cfa223 admin admin] PATH_INFO: `/v3/projects/9e6d1797f6e64aa89c5b31530a6e30ae/users/1bf441f7eed44a05b8e1d8ee756f3f49/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:10.993665 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b767e11-fe65-4efd-bd8d-4ae6a7cfa223 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e6d1797f6e64aa89c5b31530a6e30ae, user_id=1bf441f7eed44a05b8e1d8ee756f3f49, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:10.997288 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b767e11-fe65-4efd-bd8d-4ae6a7cfa223 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:11.017104 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 771/1552] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:34:10 2022] PUT /identity/v3/projects/9e6d1797f6e64aa89c5b31530a6e30ae/users/1bf441f7eed44a05b8e1d8ee756f3f49/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:34:11.028536 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a45e5-fad1-4c27-b0ac-6bc9fe3488b8 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:11.028922 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a45e5-fad1-4c27-b0ac-6bc9fe3488b8 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:11.029111 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01a45e5-fad1-4c27-b0ac-6bc9fe3488b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:11.064719 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-f01a45e5-fad1-4c27-b0ac-6bc9fe3488b8 None None] MFA Rules not processed for user `1bf441f7eed44a05b8e1d8ee756f3f49`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:34:11.066285 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f01a45e5-fad1-4c27-b0ac-6bc9fe3488b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:11.180586 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 782/1553] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:34:11 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:34:11.192926 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b767e11-fe65-4efd-bd8d-4ae6a7cfa223 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:11.196147 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2b767e11-fe65-4efd-bd8d-4ae6a7cfa223 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:11.316239 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b767e11-fe65-4efd-bd8d-4ae6a7cfa223 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:11.321066 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984e2e34-36b3-43d5-849e-f731348a3e9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:11.322054 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e2e34-36b3-43d5-849e-f731348a3e9f service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:11.322784 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e2e34-36b3-43d5-849e-f731348a3e9f service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:11.323206 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984e2e34-36b3-43d5-849e-f731348a3e9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:11.336558 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984e2e34-36b3-43d5-849e-f731348a3e9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:11.338996 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984e2e34-36b3-43d5-849e-f731348a3e9f service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:11.351035 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 772/1554] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:34:11 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:34:11.379064 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01a45e5-fad1-4c27-b0ac-6bc9fe3488b8 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:11.481608 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01a45e5-fad1-4c27-b0ac-6bc9fe3488b8 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:11.490361 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d087e51-9cac-4471-a1c5-8df702ae3ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:11.491398 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d087e51-9cac-4471-a1c5-8df702ae3ab0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:11.491981 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d087e51-9cac-4471-a1c5-8df702ae3ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:11.492413 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d087e51-9cac-4471-a1c5-8df702ae3ab0 admin admin] PATH_INFO: `/v3/users/1bf441f7eed44a05b8e1d8ee756f3f49` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:11.494550 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d087e51-9cac-4471-a1c5-8df702ae3ab0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bf441f7eed44a05b8e1d8ee756f3f49)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:11.498308 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d087e51-9cac-4471-a1c5-8df702ae3ab0 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:11.611371 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-4d087e51-9cac-4471-a1c5-8df702ae3ab0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bf441f7eed44a05b8e1d8ee756f3f49'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:11.612498 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-4d087e51-9cac-4471-a1c5-8df702ae3ab0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bf441f7eed44a05b8e1d8ee756f3f49'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:11.614057 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-4d087e51-9cac-4471-a1c5-8df702ae3ab0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bf441f7eed44a05b8e1d8ee756f3f49'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:11.635497 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-4d087e51-9cac-4471-a1c5-8df702ae3ab0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bf441f7eed44a05b8e1d8ee756f3f49'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:11.653844 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-4d087e51-9cac-4471-a1c5-8df702ae3ab0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bf441f7eed44a05b8e1d8ee756f3f49'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:11.670844 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 783/1555] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 03:34:11 2022] DELETE /identity/v3/users/1bf441f7eed44a05b8e1d8ee756f3f49 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 03:34:11.675249 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76fd12d-0a76-422e-83e1-dc78855f9ab8 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:11.675785 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76fd12d-0a76-422e-83e1-dc78855f9ab8 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:11.676331 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76fd12d-0a76-422e-83e1-dc78855f9ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:11.700248 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-f76fd12d-0a76-422e-83e1-dc78855f9ab8 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:34:11.702698 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f76fd12d-0a76-422e-83e1-dc78855f9ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:11.744108 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:34:11.784228 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 773/1556] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:34:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:34:11.793917 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d087e51-9cac-4471-a1c5-8df702ae3ab0 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:11.795331 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-4d087e51-9cac-4471-a1c5-8df702ae3ab0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:11.806096 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:34:11.861654 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d087e51-9cac-4471-a1c5-8df702ae3ab0 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:11.869328 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83107209-be4b-4395-af37-237087b2d844 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:11.870351 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83107209-be4b-4395-af37-237087b2d844 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:11.870790 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83107209-be4b-4395-af37-237087b2d844 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:11.871070 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83107209-be4b-4395-af37-237087b2d844 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:11.884607 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83107209-be4b-4395-af37-237087b2d844 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:11.886076 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83107209-be4b-4395-af37-237087b2d844 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:11.904225 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 784/1557] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:34:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:34:12.033924 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76fd12d-0a76-422e-83e1-dc78855f9ab8 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:12.033924 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f76fd12d-0a76-422e-83e1-dc78855f9ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:12.057278 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76fd12d-0a76-422e-83e1-dc78855f9ab8 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:12.081161 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5bb64a-f3fe-431a-b086-558403d234d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:12.082004 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5bb64a-f3fe-431a-b086-558403d234d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:12.082467 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5bb64a-f3fe-431a-b086-558403d234d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:12.082735 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5bb64a-f3fe-431a-b086-558403d234d2 admin admin] PATH_INFO: `/v3/users/263f27b53fcd4549bfc07c72c3d63ee6` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:12.084182 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5bb64a-f3fe-431a-b086-558403d234d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=263f27b53fcd4549bfc07c72c3d63ee6)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:12.086125 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5bb64a-f3fe-431a-b086-558403d234d2 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:12.160712 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1d5bb64a-f3fe-431a-b086-558403d234d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '263f27b53fcd4549bfc07c72c3d63ee6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:12.171868 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1d5bb64a-f3fe-431a-b086-558403d234d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '263f27b53fcd4549bfc07c72c3d63ee6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:12.173572 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1d5bb64a-f3fe-431a-b086-558403d234d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '263f27b53fcd4549bfc07c72c3d63ee6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:12.190663 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1d5bb64a-f3fe-431a-b086-558403d234d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '263f27b53fcd4549bfc07c72c3d63ee6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:12.200886 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1d5bb64a-f3fe-431a-b086-558403d234d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '263f27b53fcd4549bfc07c72c3d63ee6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:12.215276 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 774/1558] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 03:34:12 2022] DELETE /identity/v3/users/263f27b53fcd4549bfc07c72c3d63ee6 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:34:12.222542 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28a80df-6edd-4779-babd-537b05c4bcda None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:12.222542 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28a80df-6edd-4779-babd-537b05c4bcda None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:12.222542 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28a80df-6edd-4779-babd-537b05c4bcda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:12.266633 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-f28a80df-6edd-4779-babd-537b05c4bcda None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:34:12.267361 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f28a80df-6edd-4779-babd-537b05c4bcda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:12.384805 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 785/1559] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:34:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:34:12.396163 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5bb64a-f3fe-431a-b086-558403d234d2 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:12.397573 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1d5bb64a-f3fe-431a-b086-558403d234d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:12.468075 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5bb64a-f3fe-431a-b086-558403d234d2 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:12.472148 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d87cbb8-d131-4dc3-9826-397878f24cce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:12.473064 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d87cbb8-d131-4dc3-9826-397878f24cce service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:12.473064 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d87cbb8-d131-4dc3-9826-397878f24cce service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:12.473334 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d87cbb8-d131-4dc3-9826-397878f24cce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:12.483781 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d87cbb8-d131-4dc3-9826-397878f24cce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:12.485787 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d87cbb8-d131-4dc3-9826-397878f24cce service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:12.496750 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 775/1560] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:34:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:34:12.859601 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28a80df-6edd-4779-babd-537b05c4bcda None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:12.862242 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f28a80df-6edd-4779-babd-537b05c4bcda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:12.877450 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28a80df-6edd-4779-babd-537b05c4bcda None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:12.882905 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dee59a-8b0f-45da-8c0f-27a81e883cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:12.884018 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dee59a-8b0f-45da-8c0f-27a81e883cb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:12.884358 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dee59a-8b0f-45da-8c0f-27a81e883cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:12.884635 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dee59a-8b0f-45da-8c0f-27a81e883cb0 admin admin] PATH_INFO: `/v3/projects/9e6d1797f6e64aa89c5b31530a6e30ae` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:12.885950 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dee59a-8b0f-45da-8c0f-27a81e883cb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e6d1797f6e64aa89c5b31530a6e30ae)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:12.889148 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dee59a-8b0f-45da-8c0f-27a81e883cb0 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:12.963982 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e9dee59a-8b0f-45da-8c0f-27a81e883cb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e6d1797f6e64aa89c5b31530a6e30ae', 'request_id': 'req-e9dee59a-8b0f-45da-8c0f-27a81e883cb0'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:12.971423 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e9dee59a-8b0f-45da-8c0f-27a81e883cb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e6d1797f6e64aa89c5b31530a6e30ae', 'request_id': 'req-e9dee59a-8b0f-45da-8c0f-27a81e883cb0'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:12.979617 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e9dee59a-8b0f-45da-8c0f-27a81e883cb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e6d1797f6e64aa89c5b31530a6e30ae', 'request_id': 'req-e9dee59a-8b0f-45da-8c0f-27a81e883cb0'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:12.996968 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e9dee59a-8b0f-45da-8c0f-27a81e883cb0 admin admin] The token cache is being invalidate because project 9e6d1797f6e64aa89c5b31530a6e30ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 03:34:12.998056 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e9dee59a-8b0f-45da-8c0f-27a81e883cb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:12.998612 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e9dee59a-8b0f-45da-8c0f-27a81e883cb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:13.001855 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 786/1561] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 03:34:12 2022] DELETE /identity/v3/projects/9e6d1797f6e64aa89c5b31530a6e30ae => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:34:13.267336 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d87cbb8-d131-4dc3-9826-397878f24cce service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:13.270911 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2d87cbb8-d131-4dc3-9826-397878f24cce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:13.322092 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d87cbb8-d131-4dc3-9826-397878f24cce service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:13.399435 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:34:13.416957 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f575b2-3a00-4692-9da6-c3c215a7fd66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:13.418118 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f575b2-3a00-4692-9da6-c3c215a7fd66 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:13.418852 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f575b2-3a00-4692-9da6-c3c215a7fd66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:13.419140 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f575b2-3a00-4692-9da6-c3c215a7fd66 admin admin] PATH_INFO: `/v3/projects/95dfa33ba0fd4fd889a8ab63b6f573da` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:13.421057 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f575b2-3a00-4692-9da6-c3c215a7fd66 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95dfa33ba0fd4fd889a8ab63b6f573da)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:13.425228 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f575b2-3a00-4692-9da6-c3c215a7fd66 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:13.488044 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-78f575b2-3a00-4692-9da6-c3c215a7fd66 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95dfa33ba0fd4fd889a8ab63b6f573da', 'request_id': 'req-78f575b2-3a00-4692-9da6-c3c215a7fd66'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:13.497168 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-78f575b2-3a00-4692-9da6-c3c215a7fd66 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95dfa33ba0fd4fd889a8ab63b6f573da', 'request_id': 'req-78f575b2-3a00-4692-9da6-c3c215a7fd66'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:13.506626 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-78f575b2-3a00-4692-9da6-c3c215a7fd66 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95dfa33ba0fd4fd889a8ab63b6f573da', 'request_id': 'req-78f575b2-3a00-4692-9da6-c3c215a7fd66'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:13.521542 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-78f575b2-3a00-4692-9da6-c3c215a7fd66 admin admin] The token cache is being invalidate because project 95dfa33ba0fd4fd889a8ab63b6f573da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 03:34:13.521699 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-78f575b2-3a00-4692-9da6-c3c215a7fd66 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:13.522542 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-78f575b2-3a00-4692-9da6-c3c215a7fd66 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:13.525322 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 776/1562] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 03:34:13 2022] DELETE /identity/v3/projects/95dfa33ba0fd4fd889a8ab63b6f573da => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:34:13.530722 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dee59a-8b0f-45da-8c0f-27a81e883cb0 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:13.539450 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e9dee59a-8b0f-45da-8c0f-27a81e883cb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:13.556465 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dee59a-8b0f-45da-8c0f-27a81e883cb0 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:13.598287 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:34:13.632160 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc31d5c2-a5bb-407c-b0ad-0eb0081f6dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:13.632903 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc31d5c2-a5bb-407c-b0ad-0eb0081f6dc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:13.633077 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc31d5c2-a5bb-407c-b0ad-0eb0081f6dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:13.633907 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc31d5c2-a5bb-407c-b0ad-0eb0081f6dc3 admin admin] PATH_INFO: `/v3/users/21c2ee4d66ee4cbdacc52596ebcf95d9` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:13.636727 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc31d5c2-a5bb-407c-b0ad-0eb0081f6dc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21c2ee4d66ee4cbdacc52596ebcf95d9)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:13.639033 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc31d5c2-a5bb-407c-b0ad-0eb0081f6dc3 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:13.714752 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-fc31d5c2-a5bb-407c-b0ad-0eb0081f6dc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21c2ee4d66ee4cbdacc52596ebcf95d9'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:13.715417 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-fc31d5c2-a5bb-407c-b0ad-0eb0081f6dc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21c2ee4d66ee4cbdacc52596ebcf95d9'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:13.716923 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-fc31d5c2-a5bb-407c-b0ad-0eb0081f6dc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21c2ee4d66ee4cbdacc52596ebcf95d9'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:13.759542 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-fc31d5c2-a5bb-407c-b0ad-0eb0081f6dc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21c2ee4d66ee4cbdacc52596ebcf95d9'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:13.781200 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-fc31d5c2-a5bb-407c-b0ad-0eb0081f6dc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21c2ee4d66ee4cbdacc52596ebcf95d9'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:13.797114 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 787/1563] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 03:34:13 2022] DELETE /identity/v3/users/21c2ee4d66ee4cbdacc52596ebcf95d9 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:34:14.089349 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f575b2-3a00-4692-9da6-c3c215a7fd66 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:14.090840 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-78f575b2-3a00-4692-9da6-c3c215a7fd66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:14.122364 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f575b2-3a00-4692-9da6-c3c215a7fd66 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:14.201332 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546dc9dc-0daf-456d-83cb-6dac67afea4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:14.208323 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546dc9dc-0daf-456d-83cb-6dac67afea4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:14.209005 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546dc9dc-0daf-456d-83cb-6dac67afea4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:14.209414 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546dc9dc-0daf-456d-83cb-6dac67afea4e admin admin] PATH_INFO: `/v3/projects/7a547cfa2eaa444b9a7234fc2555779e` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:14.210750 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546dc9dc-0daf-456d-83cb-6dac67afea4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a547cfa2eaa444b9a7234fc2555779e)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:14.212751 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546dc9dc-0daf-456d-83cb-6dac67afea4e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:14.316918 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-546dc9dc-0daf-456d-83cb-6dac67afea4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a547cfa2eaa444b9a7234fc2555779e', 'request_id': 'req-546dc9dc-0daf-456d-83cb-6dac67afea4e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:14.328866 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-546dc9dc-0daf-456d-83cb-6dac67afea4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a547cfa2eaa444b9a7234fc2555779e', 'request_id': 'req-546dc9dc-0daf-456d-83cb-6dac67afea4e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:14.338124 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-546dc9dc-0daf-456d-83cb-6dac67afea4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a547cfa2eaa444b9a7234fc2555779e', 'request_id': 'req-546dc9dc-0daf-456d-83cb-6dac67afea4e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:14.355695 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-546dc9dc-0daf-456d-83cb-6dac67afea4e admin admin] The token cache is being invalidate because project 7a547cfa2eaa444b9a7234fc2555779e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 03:34:14.356201 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-546dc9dc-0daf-456d-83cb-6dac67afea4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:14.357948 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-546dc9dc-0daf-456d-83cb-6dac67afea4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:14.360606 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 777/1564] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 03:34:14 2022] DELETE /identity/v3/projects/7a547cfa2eaa444b9a7234fc2555779e => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:34:14.365806 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e6e2d2-0f7c-4adf-bb82-c856c03de90e None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:14.366181 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e6e2d2-0f7c-4adf-bb82-c856c03de90e None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:14.366532 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e6e2d2-0f7c-4adf-bb82-c856c03de90e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:14.392921 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-01e6e2d2-0f7c-4adf-bb82-c856c03de90e None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:34:14.394218 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-01e6e2d2-0f7c-4adf-bb82-c856c03de90e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:14.480006 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 788/1565] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:34:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:34:14.488901 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f6ab58-29fc-4132-89a3-1bc8e7fd7ecb None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:14.490461 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f6ab58-29fc-4132-89a3-1bc8e7fd7ecb None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:14.490993 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f6ab58-29fc-4132-89a3-1bc8e7fd7ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:14.511425 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:34:14.520339 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-15f6ab58-29fc-4132-89a3-1bc8e7fd7ecb None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:34:14.545771 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-15f6ab58-29fc-4132-89a3-1bc8e7fd7ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:14.555422 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 778/1566] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:34:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:34:14.560024 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e6e2d2-0f7c-4adf-bb82-c856c03de90e None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:14.570948 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e6e2d2-0f7c-4adf-bb82-c856c03de90e None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:14.576009 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ce0f68-b594-4080-a88d-2ca36c01f1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:14.576697 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ce0f68-b594-4080-a88d-2ca36c01f1bf admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:14.577042 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ce0f68-b594-4080-a88d-2ca36c01f1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:14.577388 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ce0f68-b594-4080-a88d-2ca36c01f1bf admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:14.578121 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ce0f68-b594-4080-a88d-2ca36c01f1bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:34:14.578536 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ce0f68-b594-4080-a88d-2ca36c01f1bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:14.580950 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ce0f68-b594-4080-a88d-2ca36c01f1bf admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:14.594441 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 789/1567] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 03:34:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:34:14.598909 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f6ab58-29fc-4132-89a3-1bc8e7fd7ecb None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:14.608758 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f6ab58-29fc-4132-89a3-1bc8e7fd7ecb None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:14.613929 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbb0cde-6dfd-40b6-a7b2-2141e86a1a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:14.614564 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb0cde-6dfd-40b6-a7b2-2141e86a1a57 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:14.614845 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb0cde-6dfd-40b6-a7b2-2141e86a1a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:14.615120 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb0cde-6dfd-40b6-a7b2-2141e86a1a57 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:14.615766 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbb0cde-6dfd-40b6-a7b2-2141e86a1a57 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:14.617716 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbb0cde-6dfd-40b6-a7b2-2141e86a1a57 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:14.652019 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 779/1568] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:34:14 2022] POST /identity/v3/projects => generated 369 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:34:14.656113 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ce0f68-b594-4080-a88d-2ca36c01f1bf admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:14.668739 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ce0f68-b594-4080-a88d-2ca36c01f1bf admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:14.727618 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:34:14.747356 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5048940-ecc5-47d3-80e7-40f583e759fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:14.748371 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5048940-ecc5-47d3-80e7-40f583e759fb admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:14.748690 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5048940-ecc5-47d3-80e7-40f583e759fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:14.748965 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5048940-ecc5-47d3-80e7-40f583e759fb admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:14.749614 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5048940-ecc5-47d3-80e7-40f583e759fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:14.751557 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5048940-ecc5-47d3-80e7-40f583e759fb admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:14.782335 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 790/1569] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:34:14 2022] POST /identity/v3/users => generated 285 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 03:34:14.783967 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbb0cde-6dfd-40b6-a7b2-2141e86a1a57 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:14.794737 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbb0cde-6dfd-40b6-a7b2-2141e86a1a57 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:14.800273 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd8e693-36a7-492b-bcdb-c81de10bd5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:14.800914 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd8e693-36a7-492b-bcdb-c81de10bd5e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:14.801194 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd8e693-36a7-492b-bcdb-c81de10bd5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:14.801463 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd8e693-36a7-492b-bcdb-c81de10bd5e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:14.802115 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd8e693-36a7-492b-bcdb-c81de10bd5e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:14.803810 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd8e693-36a7-492b-bcdb-c81de10bd5e0 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:14.814439 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 780/1570] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:34:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:34:14.818487 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5048940-ecc5-47d3-80e7-40f583e759fb admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:14.829081 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5048940-ecc5-47d3-80e7-40f583e759fb admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:14.833587 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f696b883-961f-4e19-82ed-140eab0fc21c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:14.834187 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f696b883-961f-4e19-82ed-140eab0fc21c admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:14.834475 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f696b883-961f-4e19-82ed-140eab0fc21c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:14.834749 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f696b883-961f-4e19-82ed-140eab0fc21c admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:14.835388 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f696b883-961f-4e19-82ed-140eab0fc21c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:14.837004 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f696b883-961f-4e19-82ed-140eab0fc21c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:14.848602 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 791/1571] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:34:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:34:14.853009 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd8e693-36a7-492b-bcdb-c81de10bd5e0 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:14.863189 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd8e693-36a7-492b-bcdb-c81de10bd5e0 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:14.868878 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd2fec9-bf43-4ed1-8c0d-828753fc70ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:14.869570 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd2fec9-bf43-4ed1-8c0d-828753fc70ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:14.869854 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd2fec9-bf43-4ed1-8c0d-828753fc70ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:14.870129 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd2fec9-bf43-4ed1-8c0d-828753fc70ef admin admin] PATH_INFO: `/v3/projects/4ab92d9e684348ddba164fe89031586d/users/96ce6213a8ea4cea9117b0b468716db1/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:14.886757 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd2fec9-bf43-4ed1-8c0d-828753fc70ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ab92d9e684348ddba164fe89031586d, user_id=96ce6213a8ea4cea9117b0b468716db1, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:14.889441 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd2fec9-bf43-4ed1-8c0d-828753fc70ef admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:14.902654 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 781/1572] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:34:14 2022] PUT /identity/v3/projects/4ab92d9e684348ddba164fe89031586d/users/96ce6213a8ea4cea9117b0b468716db1/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:34:14.913082 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9cedec-c3ef-42f6-8de5-38257c0a96bb None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:14.913512 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9cedec-c3ef-42f6-8de5-38257c0a96bb None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:14.913856 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9cedec-c3ef-42f6-8de5-38257c0a96bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:14.947498 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-ed9cedec-c3ef-42f6-8de5-38257c0a96bb None None] MFA Rules not processed for user `96ce6213a8ea4cea9117b0b468716db1`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:34:14.950107 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ed9cedec-c3ef-42f6-8de5-38257c0a96bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:15.029809 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 792/1573] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:34:14 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:34:15.034518 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd2fec9-bf43-4ed1-8c0d-828753fc70ef admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:15.107687 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd2fec9-bf43-4ed1-8c0d-828753fc70ef admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:15.112710 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471ca9d5-de0e-4f1f-9318-b2f6722b531f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:15.114006 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471ca9d5-de0e-4f1f-9318-b2f6722b531f admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:15.114006 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471ca9d5-de0e-4f1f-9318-b2f6722b531f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:15.114006 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471ca9d5-de0e-4f1f-9318-b2f6722b531f admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:15.114485 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471ca9d5-de0e-4f1f-9318-b2f6722b531f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:15.116638 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471ca9d5-de0e-4f1f-9318-b2f6722b531f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:15.166540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 782/1574] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:34:15 2022] POST /identity/v3/projects => generated 367 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:34:15.170715 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9cedec-c3ef-42f6-8de5-38257c0a96bb None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:15.193142 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9cedec-c3ef-42f6-8de5-38257c0a96bb None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:15.259769 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5bf43e-a21c-4f3c-b7e1-4c27f5c41d09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:15.260505 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5bf43e-a21c-4f3c-b7e1-4c27f5c41d09 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:15.260810 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5bf43e-a21c-4f3c-b7e1-4c27f5c41d09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:15.261088 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5bf43e-a21c-4f3c-b7e1-4c27f5c41d09 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:15.261735 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5bf43e-a21c-4f3c-b7e1-4c27f5c41d09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:15.263737 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5bf43e-a21c-4f3c-b7e1-4c27f5c41d09 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:15.294395 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 793/1575] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:34:15 2022] POST /identity/v3/users => generated 290 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 03:34:15.295638 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471ca9d5-de0e-4f1f-9318-b2f6722b531f admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:15.306335 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471ca9d5-de0e-4f1f-9318-b2f6722b531f admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:15.311071 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4983fab-207b-4a03-b0f3-b8fede8dfe93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:15.311679 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4983fab-207b-4a03-b0f3-b8fede8dfe93 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:15.311983 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4983fab-207b-4a03-b0f3-b8fede8dfe93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:15.312264 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4983fab-207b-4a03-b0f3-b8fede8dfe93 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:15.313435 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4983fab-207b-4a03-b0f3-b8fede8dfe93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:15.315035 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4983fab-207b-4a03-b0f3-b8fede8dfe93 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:15.325713 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 783/1576] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:34:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:34:15.330284 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5bf43e-a21c-4f3c-b7e1-4c27f5c41d09 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:15.342204 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5bf43e-a21c-4f3c-b7e1-4c27f5c41d09 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:15.348634 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b446541-cdfb-4757-bd74-c828e3d2b5ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:15.349320 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b446541-cdfb-4757-bd74-c828e3d2b5ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:15.349489 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b446541-cdfb-4757-bd74-c828e3d2b5ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:15.349834 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b446541-cdfb-4757-bd74-c828e3d2b5ed admin admin] PATH_INFO: `/v3/projects/0d64a23a46504b9ab4c1f7762a5aa0fb/users/592f89d9847241fcb5a3bbccb72a57f9/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:15.367615 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b446541-cdfb-4757-bd74-c828e3d2b5ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d64a23a46504b9ab4c1f7762a5aa0fb, user_id=592f89d9847241fcb5a3bbccb72a57f9, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:15.370716 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b446541-cdfb-4757-bd74-c828e3d2b5ed admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:15.386625 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 794/1577] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:34:15 2022] PUT /identity/v3/projects/0d64a23a46504b9ab4c1f7762a5aa0fb/users/592f89d9847241fcb5a3bbccb72a57f9/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:34:15.390455 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4983fab-207b-4a03-b0f3-b8fede8dfe93 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:15.402110 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4983fab-207b-4a03-b0f3-b8fede8dfe93 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:15.468014 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550a587e-e91f-4bfb-8334-7f1885c3b43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:15.468644 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550a587e-e91f-4bfb-8334-7f1885c3b43f admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:15.468916 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550a587e-e91f-4bfb-8334-7f1885c3b43f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:15.469241 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550a587e-e91f-4bfb-8334-7f1885c3b43f admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:15.469896 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550a587e-e91f-4bfb-8334-7f1885c3b43f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:15.471492 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550a587e-e91f-4bfb-8334-7f1885c3b43f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:15.482423 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 784/1578] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:34:15 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 03:34:15.487168 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b446541-cdfb-4757-bd74-c828e3d2b5ed admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:15.507519 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b446541-cdfb-4757-bd74-c828e3d2b5ed admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:15.514038 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9cd73b-8610-4f2a-9dad-7c5d4f55e41e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:15.514810 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9cd73b-8610-4f2a-9dad-7c5d4f55e41e admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:15.515175 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9cd73b-8610-4f2a-9dad-7c5d4f55e41e admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:15.515634 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9cd73b-8610-4f2a-9dad-7c5d4f55e41e admin admin] PATH_INFO: `/v3/projects/0d64a23a46504b9ab4c1f7762a5aa0fb/users/592f89d9847241fcb5a3bbccb72a57f9/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:15.518515 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9cd73b-8610-4f2a-9dad-7c5d4f55e41e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d64a23a46504b9ab4c1f7762a5aa0fb, user_id=592f89d9847241fcb5a3bbccb72a57f9, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:15.524530 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9cd73b-8610-4f2a-9dad-7c5d4f55e41e admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:15.546567 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539184b7-25ec-426a-83d5-1752ca617573 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:15.546884 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539184b7-25ec-426a-83d5-1752ca617573 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:15.547141 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539184b7-25ec-426a-83d5-1752ca617573 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:15.552707 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 795/1579] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:34:15 2022] PUT /identity/v3/projects/0d64a23a46504b9ab4c1f7762a5aa0fb/users/592f89d9847241fcb5a3bbccb72a57f9/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:34:15.602442 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:34:15.611874 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-539184b7-25ec-426a-83d5-1752ca617573 None None] MFA Rules not processed for user `592f89d9847241fcb5a3bbccb72a57f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:34:15.616595 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-539184b7-25ec-426a-83d5-1752ca617573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:15.720812 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 785/1580] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:34:15 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:34:15.731819 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9cd73b-8610-4f2a-9dad-7c5d4f55e41e admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:15.733794 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-af9cd73b-8610-4f2a-9dad-7c5d4f55e41e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:15.739916 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:34:15.803168 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9cd73b-8610-4f2a-9dad-7c5d4f55e41e admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:15.806805 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47720ff7-a94d-471a-9bb2-733f7bd4f636 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:15.807496 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47720ff7-a94d-471a-9bb2-733f7bd4f636 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:15.807783 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47720ff7-a94d-471a-9bb2-733f7bd4f636 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:15.808095 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47720ff7-a94d-471a-9bb2-733f7bd4f636 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:15.818467 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47720ff7-a94d-471a-9bb2-733f7bd4f636 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:15.819909 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47720ff7-a94d-471a-9bb2-733f7bd4f636 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:15.830020 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 796/1581] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:34:15 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:34:15.916683 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539184b7-25ec-426a-83d5-1752ca617573 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:15.920238 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-539184b7-25ec-426a-83d5-1752ca617573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:15.938507 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539184b7-25ec-426a-83d5-1752ca617573 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:15.943813 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de54c758-23e5-4dbf-aac9-86b840afcc5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:15.944628 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de54c758-23e5-4dbf-aac9-86b840afcc5f service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:15.944914 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de54c758-23e5-4dbf-aac9-86b840afcc5f service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:15.945187 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de54c758-23e5-4dbf-aac9-86b840afcc5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:16.040818 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de54c758-23e5-4dbf-aac9-86b840afcc5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:16.042671 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de54c758-23e5-4dbf-aac9-86b840afcc5f service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:16.055386 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 786/1582] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:34:15 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:34:17.642219 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa55677-dda1-41b0-8b62-a7f534dbde86 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:17.643045 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa55677-dda1-41b0-8b62-a7f534dbde86 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:17.643497 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa55677-dda1-41b0-8b62-a7f534dbde86 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:17.645488 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 797/1583] 10.222.0.26 () {54 vars in 921 bytes} [Wed Nov 2 03:34:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:34:17.664189 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29b11b1-ef74-420d-b334-2048c7cab697 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:17.664667 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29b11b1-ef74-420d-b334-2048c7cab697 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:17.668271 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29b11b1-ef74-420d-b334-2048c7cab697 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:17.706641 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-b29b11b1-ef74-420d-b334-2048c7cab697 None None] MFA Rules not processed for user `44961b06151c4da382fcd54dd1f1e146`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:34:17.709169 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b29b11b1-ef74-420d-b334-2048c7cab697 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:17.854468 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 787/1584] 10.222.0.26 () {58 vars in 1018 bytes} [Wed Nov 2 03:34:17 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:34:17.907182 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa55677-dda1-41b0-8b62-a7f534dbde86 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:17.908961 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-3fa55677-dda1-41b0-8b62-a7f534dbde86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:17.937325 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa55677-dda1-41b0-8b62-a7f534dbde86 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:17.943041 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43ed573-d4d9-4168-b14b-da402160c369 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:17.943802 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ed573-d4d9-4168-b14b-da402160c369 service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:17.944662 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ed573-d4d9-4168-b14b-da402160c369 service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:17.945014 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43ed573-d4d9-4168-b14b-da402160c369 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:17.955397 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43ed573-d4d9-4168-b14b-da402160c369 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:17.957038 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43ed573-d4d9-4168-b14b-da402160c369 service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:17.970169 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 798/1585] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:34:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:34:19.932823 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29b11b1-ef74-420d-b334-2048c7cab697 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:19.991644 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:34:20.046037 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29b11b1-ef74-420d-b334-2048c7cab697 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:20.054277 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcf366d-176e-472c-ad1a-39efe8b3ce0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:20.055303 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcf366d-176e-472c-ad1a-39efe8b3ce0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:20.055878 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcf366d-176e-472c-ad1a-39efe8b3ce0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:20.056882 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcf366d-176e-472c-ad1a-39efe8b3ce0a admin admin] PATH_INFO: `/v3/users/96ce6213a8ea4cea9117b0b468716db1` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:20.058043 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcf366d-176e-472c-ad1a-39efe8b3ce0a admin admin] RBAC: Authorizing `identity:delete_user(user_id=96ce6213a8ea4cea9117b0b468716db1)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:20.061996 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcf366d-176e-472c-ad1a-39efe8b3ce0a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:20.113840 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-fbcf366d-176e-472c-ad1a-39efe8b3ce0a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96ce6213a8ea4cea9117b0b468716db1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:20.120906 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-fbcf366d-176e-472c-ad1a-39efe8b3ce0a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96ce6213a8ea4cea9117b0b468716db1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:20.121738 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-fbcf366d-176e-472c-ad1a-39efe8b3ce0a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96ce6213a8ea4cea9117b0b468716db1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:20.134943 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-fbcf366d-176e-472c-ad1a-39efe8b3ce0a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96ce6213a8ea4cea9117b0b468716db1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:20.142883 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-fbcf366d-176e-472c-ad1a-39efe8b3ce0a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96ce6213a8ea4cea9117b0b468716db1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:20.149554 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 788/1586] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 03:34:19 2022] DELETE /identity/v3/users/96ce6213a8ea4cea9117b0b468716db1 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:34:20.153963 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b139aa7-d463-407e-a599-cf05e68395af None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:20.154458 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b139aa7-d463-407e-a599-cf05e68395af None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:20.154534 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b139aa7-d463-407e-a599-cf05e68395af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:20.174676 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-1b139aa7-d463-407e-a599-cf05e68395af None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:34:20.175894 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-1b139aa7-d463-407e-a599-cf05e68395af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:20.290382 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 799/1587] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:34:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:34:20.313459 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcf366d-176e-472c-ad1a-39efe8b3ce0a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:20.315125 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-fbcf366d-176e-472c-ad1a-39efe8b3ce0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:20.425004 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcf366d-176e-472c-ad1a-39efe8b3ce0a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:20.431693 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4654c8-458b-4d3e-a1be-1a4bb80727a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:20.432596 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4654c8-458b-4d3e-a1be-1a4bb80727a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:20.433200 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4654c8-458b-4d3e-a1be-1a4bb80727a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:20.433350 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4654c8-458b-4d3e-a1be-1a4bb80727a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:20.444531 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4654c8-458b-4d3e-a1be-1a4bb80727a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:20.446620 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4654c8-458b-4d3e-a1be-1a4bb80727a1 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:20.473951 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 789/1588] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:34:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:34:20.830628 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b139aa7-d463-407e-a599-cf05e68395af None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:20.833082 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-1b139aa7-d463-407e-a599-cf05e68395af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:20.847049 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b139aa7-d463-407e-a599-cf05e68395af None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:20.851927 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08142eb0-b9b5-4083-a9e0-7d7976c6e359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:20.852666 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08142eb0-b9b5-4083-a9e0-7d7976c6e359 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:20.852956 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08142eb0-b9b5-4083-a9e0-7d7976c6e359 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:20.853233 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08142eb0-b9b5-4083-a9e0-7d7976c6e359 admin admin] PATH_INFO: `/v3/projects/4ab92d9e684348ddba164fe89031586d` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:20.854811 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08142eb0-b9b5-4083-a9e0-7d7976c6e359 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ab92d9e684348ddba164fe89031586d)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:20.857376 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08142eb0-b9b5-4083-a9e0-7d7976c6e359 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:20.926919 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-08142eb0-b9b5-4083-a9e0-7d7976c6e359 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ab92d9e684348ddba164fe89031586d', 'request_id': 'req-08142eb0-b9b5-4083-a9e0-7d7976c6e359'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:20.937606 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-08142eb0-b9b5-4083-a9e0-7d7976c6e359 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ab92d9e684348ddba164fe89031586d', 'request_id': 'req-08142eb0-b9b5-4083-a9e0-7d7976c6e359'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:20.945855 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-08142eb0-b9b5-4083-a9e0-7d7976c6e359 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ab92d9e684348ddba164fe89031586d', 'request_id': 'req-08142eb0-b9b5-4083-a9e0-7d7976c6e359'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:20.965683 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-08142eb0-b9b5-4083-a9e0-7d7976c6e359 admin admin] The token cache is being invalidate because project 4ab92d9e684348ddba164fe89031586d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 03:34:20.966083 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-08142eb0-b9b5-4083-a9e0-7d7976c6e359 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:20.966953 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-08142eb0-b9b5-4083-a9e0-7d7976c6e359 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:20.971728 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 800/1589] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 03:34:20 2022] DELETE /identity/v3/projects/4ab92d9e684348ddba164fe89031586d => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:34:20.973049 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4654c8-458b-4d3e-a1be-1a4bb80727a1 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:20.988006 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4654c8-458b-4d3e-a1be-1a4bb80727a1 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:21.067781 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:34:21.088394 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e172415-1ddc-4b18-a27e-b88a962e6632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:21.089082 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e172415-1ddc-4b18-a27e-b88a962e6632 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:21.089361 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e172415-1ddc-4b18-a27e-b88a962e6632 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:21.090358 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e172415-1ddc-4b18-a27e-b88a962e6632 admin admin] PATH_INFO: `/v3/users/592f89d9847241fcb5a3bbccb72a57f9` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:21.092813 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e172415-1ddc-4b18-a27e-b88a962e6632 admin admin] RBAC: Authorizing `identity:delete_user(user_id=592f89d9847241fcb5a3bbccb72a57f9)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:21.098394 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e172415-1ddc-4b18-a27e-b88a962e6632 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:21.166792 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-3e172415-1ddc-4b18-a27e-b88a962e6632 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '592f89d9847241fcb5a3bbccb72a57f9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:21.177522 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-3e172415-1ddc-4b18-a27e-b88a962e6632 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '592f89d9847241fcb5a3bbccb72a57f9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:21.178272 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-3e172415-1ddc-4b18-a27e-b88a962e6632 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '592f89d9847241fcb5a3bbccb72a57f9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:21.194637 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-3e172415-1ddc-4b18-a27e-b88a962e6632 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '592f89d9847241fcb5a3bbccb72a57f9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:21.206021 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-3e172415-1ddc-4b18-a27e-b88a962e6632 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '592f89d9847241fcb5a3bbccb72a57f9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:21.216165 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 790/1590] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 03:34:20 2022] DELETE /identity/v3/users/592f89d9847241fcb5a3bbccb72a57f9 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:34:21.692207 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08142eb0-b9b5-4083-a9e0-7d7976c6e359 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:21.693741 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-08142eb0-b9b5-4083-a9e0-7d7976c6e359 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:21.708333 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08142eb0-b9b5-4083-a9e0-7d7976c6e359 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:21.736052 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:34:21.798565 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a3f1af-684a-43a6-9a54-ddd94a50abc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:21.799228 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a3f1af-684a-43a6-9a54-ddd94a50abc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:21.799511 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a3f1af-684a-43a6-9a54-ddd94a50abc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:21.799790 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a3f1af-684a-43a6-9a54-ddd94a50abc0 admin admin] PATH_INFO: `/v3/projects/0d64a23a46504b9ab4c1f7762a5aa0fb` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:21.801022 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a3f1af-684a-43a6-9a54-ddd94a50abc0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d64a23a46504b9ab4c1f7762a5aa0fb)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:21.802979 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a3f1af-684a-43a6-9a54-ddd94a50abc0 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:21.856293 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-91a3f1af-684a-43a6-9a54-ddd94a50abc0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d64a23a46504b9ab4c1f7762a5aa0fb', 'request_id': 'req-91a3f1af-684a-43a6-9a54-ddd94a50abc0'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:21.862625 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-91a3f1af-684a-43a6-9a54-ddd94a50abc0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d64a23a46504b9ab4c1f7762a5aa0fb', 'request_id': 'req-91a3f1af-684a-43a6-9a54-ddd94a50abc0'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:21.869887 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-91a3f1af-684a-43a6-9a54-ddd94a50abc0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d64a23a46504b9ab4c1f7762a5aa0fb', 'request_id': 'req-91a3f1af-684a-43a6-9a54-ddd94a50abc0'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:21.884252 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-91a3f1af-684a-43a6-9a54-ddd94a50abc0 admin admin] The token cache is being invalidate because project 0d64a23a46504b9ab4c1f7762a5aa0fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 03:34:21.884572 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-91a3f1af-684a-43a6-9a54-ddd94a50abc0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:21.885175 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-91a3f1af-684a-43a6-9a54-ddd94a50abc0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:21.886911 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 801/1591] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 03:34:21 2022] DELETE /identity/v3/projects/0d64a23a46504b9ab4c1f7762a5aa0fb => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:34:21.891693 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d101a80-1b3d-4faf-81c9-280472be1c13 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:21.892015 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d101a80-1b3d-4faf-81c9-280472be1c13 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:21.892298 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d101a80-1b3d-4faf-81c9-280472be1c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:21.912814 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-4d101a80-1b3d-4faf-81c9-280472be1c13 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:34:21.914151 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4d101a80-1b3d-4faf-81c9-280472be1c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:21.974649 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 791/1592] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:34:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:34:21.982479 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f9965-8b40-4a22-a508-c90838b2aefe None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:21.982766 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f9965-8b40-4a22-a508-c90838b2aefe None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:21.983035 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f9965-8b40-4a22-a508-c90838b2aefe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:22.004484 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-dc7f9965-8b40-4a22-a508-c90838b2aefe None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:34:22.007503 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-dc7f9965-8b40-4a22-a508-c90838b2aefe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:22.021127 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 802/1593] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:34:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:34:22.021692 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d101a80-1b3d-4faf-81c9-280472be1c13 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:22.033612 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d101a80-1b3d-4faf-81c9-280472be1c13 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:22.039745 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fde20d-9246-4284-b2a1-a0b210650a97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:22.040403 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fde20d-9246-4284-b2a1-a0b210650a97 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:22.040890 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fde20d-9246-4284-b2a1-a0b210650a97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:22.041169 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47fde20d-9246-4284-b2a1-a0b210650a97 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:22.041993 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fde20d-9246-4284-b2a1-a0b210650a97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:34:22.042293 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fde20d-9246-4284-b2a1-a0b210650a97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:22.043879 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47fde20d-9246-4284-b2a1-a0b210650a97 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:22.056878 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 792/1594] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 03:34:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:34:22.061319 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f9965-8b40-4a22-a508-c90838b2aefe None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:22.072994 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f9965-8b40-4a22-a508-c90838b2aefe None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:22.078028 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7970b0f-155f-448e-846b-2406e2ee42cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:22.079334 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7970b0f-155f-448e-846b-2406e2ee42cd admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:22.079658 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7970b0f-155f-448e-846b-2406e2ee42cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:22.079932 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7970b0f-155f-448e-846b-2406e2ee42cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:22.080638 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7970b0f-155f-448e-846b-2406e2ee42cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:22.082567 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7970b0f-155f-448e-846b-2406e2ee42cd admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:22.113279 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 803/1595] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:34:22 2022] POST /identity/v3/projects => generated 381 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 03:34:22.117677 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fde20d-9246-4284-b2a1-a0b210650a97 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:22.128071 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47fde20d-9246-4284-b2a1-a0b210650a97 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:22.182885 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:34:22.189991 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b950c024-2f34-4df8-80b8-b248d9f4c3d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:22.190907 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b950c024-2f34-4df8-80b8-b248d9f4c3d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:22.191255 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b950c024-2f34-4df8-80b8-b248d9f4c3d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:22.191692 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b950c024-2f34-4df8-80b8-b248d9f4c3d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:22.192729 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b950c024-2f34-4df8-80b8-b248d9f4c3d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:22.194786 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b950c024-2f34-4df8-80b8-b248d9f4c3d3 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:22.218953 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 793/1596] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:34:22 2022] POST /identity/v3/users => generated 291 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 03:34:22.223657 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7970b0f-155f-448e-846b-2406e2ee42cd admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:22.240278 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7970b0f-155f-448e-846b-2406e2ee42cd admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:22.245727 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34df28e1-ddba-4ac3-b798-d0a7c47502fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:22.248233 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df28e1-ddba-4ac3-b798-d0a7c47502fc admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:22.248519 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df28e1-ddba-4ac3-b798-d0a7c47502fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:22.248796 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34df28e1-ddba-4ac3-b798-d0a7c47502fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:22.249451 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34df28e1-ddba-4ac3-b798-d0a7c47502fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:22.251231 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34df28e1-ddba-4ac3-b798-d0a7c47502fc admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:22.264007 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 804/1597] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:34:22 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:34:22.268568 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b950c024-2f34-4df8-80b8-b248d9f4c3d3 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:22.279847 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b950c024-2f34-4df8-80b8-b248d9f4c3d3 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:22.286003 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8d16a2-ec0a-4058-af92-b15e606f26a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:22.286708 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d16a2-ec0a-4058-af92-b15e606f26a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:22.287054 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d16a2-ec0a-4058-af92-b15e606f26a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:22.287395 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8d16a2-ec0a-4058-af92-b15e606f26a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:22.288278 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8d16a2-ec0a-4058-af92-b15e606f26a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:22.289940 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8d16a2-ec0a-4058-af92-b15e606f26a4 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:22.305842 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 794/1598] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:34:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:34:22.307206 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34df28e1-ddba-4ac3-b798-d0a7c47502fc admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:22.318222 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34df28e1-ddba-4ac3-b798-d0a7c47502fc admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:22.324727 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cd2e45-2fe5-49f3-82d9-9694b722e3f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:22.325595 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd2e45-2fe5-49f3-82d9-9694b722e3f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:22.325969 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd2e45-2fe5-49f3-82d9-9694b722e3f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:22.326333 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd2e45-2fe5-49f3-82d9-9694b722e3f5 admin admin] PATH_INFO: `/v3/projects/933bd2bfa38f4dd1aa7dcec2243d2a87/users/08559931aa4546839ab7f2abe38e959d/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:22.343166 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cd2e45-2fe5-49f3-82d9-9694b722e3f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=933bd2bfa38f4dd1aa7dcec2243d2a87, user_id=08559931aa4546839ab7f2abe38e959d, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:22.345922 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cd2e45-2fe5-49f3-82d9-9694b722e3f5 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:22.359359 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 805/1599] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:34:22 2022] PUT /identity/v3/projects/933bd2bfa38f4dd1aa7dcec2243d2a87/users/08559931aa4546839ab7f2abe38e959d/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 03:34:22.370058 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f8178-f42f-4e06-b36b-f8072610efd0 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:22.370341 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f8178-f42f-4e06-b36b-f8072610efd0 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:22.370650 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f8178-f42f-4e06-b36b-f8072610efd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:22.400464 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-b99f8178-f42f-4e06-b36b-f8072610efd0 None None] MFA Rules not processed for user `08559931aa4546839ab7f2abe38e959d`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:34:22.401982 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b99f8178-f42f-4e06-b36b-f8072610efd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:22.469138 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 795/1600] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:34:22 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:34:22.473571 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cd2e45-2fe5-49f3-82d9-9694b722e3f5 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:22.520277 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:34:22.545807 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cd2e45-2fe5-49f3-82d9-9694b722e3f5 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:22.551361 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae129e7-fe0a-4d10-b379-463da6b4d555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:22.552015 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae129e7-fe0a-4d10-b379-463da6b4d555 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:22.552293 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae129e7-fe0a-4d10-b379-463da6b4d555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:22.552563 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae129e7-fe0a-4d10-b379-463da6b4d555 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:22.553205 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae129e7-fe0a-4d10-b379-463da6b4d555 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:22.555121 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae129e7-fe0a-4d10-b379-463da6b4d555 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:22.588224 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 806/1601] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:34:22 2022] POST /identity/v3/projects => generated 377 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:34:22.589805 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99f8178-f42f-4e06-b36b-f8072610efd0 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:22.601596 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99f8178-f42f-4e06-b36b-f8072610efd0 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:22.661138 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e828ca-4d50-4ae7-9d6a-cc272fc77def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:22.661766 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e828ca-4d50-4ae7-9d6a-cc272fc77def admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:22.662049 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e828ca-4d50-4ae7-9d6a-cc272fc77def admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:22.662325 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e828ca-4d50-4ae7-9d6a-cc272fc77def admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:22.662982 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e828ca-4d50-4ae7-9d6a-cc272fc77def admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:22.665169 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e828ca-4d50-4ae7-9d6a-cc272fc77def admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:22.690337 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 796/1602] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:34:22 2022] POST /identity/v3/users => generated 295 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:34:22.691492 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae129e7-fe0a-4d10-b379-463da6b4d555 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:22.702975 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae129e7-fe0a-4d10-b379-463da6b4d555 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:22.708390 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124f98b5-bbfd-4006-a114-d8de70f9d2ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:22.709014 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f98b5-bbfd-4006-a114-d8de70f9d2ea admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:22.709292 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f98b5-bbfd-4006-a114-d8de70f9d2ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:22.709562 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124f98b5-bbfd-4006-a114-d8de70f9d2ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:22.711169 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124f98b5-bbfd-4006-a114-d8de70f9d2ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:22.712140 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124f98b5-bbfd-4006-a114-d8de70f9d2ea admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:22.727859 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 807/1603] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:34:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:34:22.729112 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e828ca-4d50-4ae7-9d6a-cc272fc77def admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:22.741585 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e828ca-4d50-4ae7-9d6a-cc272fc77def admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:22.755881 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8b2b6c-19db-4ee1-b328-908e9ee895ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:22.756689 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8b2b6c-19db-4ee1-b328-908e9ee895ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:22.757011 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8b2b6c-19db-4ee1-b328-908e9ee895ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:22.757289 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8b2b6c-19db-4ee1-b328-908e9ee895ee admin admin] PATH_INFO: `/v3/projects/eeb5dbae475a486a9275519f493e496a/users/6c59cd5bcbe8488eb849f450f6023e1e/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:22.780770 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8b2b6c-19db-4ee1-b328-908e9ee895ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=eeb5dbae475a486a9275519f493e496a, user_id=6c59cd5bcbe8488eb849f450f6023e1e, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:22.784815 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8b2b6c-19db-4ee1-b328-908e9ee895ee admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:22.839373 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 797/1604] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:34:22 2022] PUT /identity/v3/projects/eeb5dbae475a486a9275519f493e496a/users/6c59cd5bcbe8488eb849f450f6023e1e/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:34:22.842838 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124f98b5-bbfd-4006-a114-d8de70f9d2ea admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:22.854074 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124f98b5-bbfd-4006-a114-d8de70f9d2ea admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:23.041954 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0ec682-7bbd-4b8f-afef-a93d5afb3ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:23.042524 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ec682-7bbd-4b8f-afef-a93d5afb3ce4 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:23.042803 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ec682-7bbd-4b8f-afef-a93d5afb3ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:23.043076 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0ec682-7bbd-4b8f-afef-a93d5afb3ce4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:23.043729 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0ec682-7bbd-4b8f-afef-a93d5afb3ce4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:23.045417 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0ec682-7bbd-4b8f-afef-a93d5afb3ce4 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:23.059136 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 808/1605] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:34:22 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 03:34:23.060575 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8b2b6c-19db-4ee1-b328-908e9ee895ee admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:23.084018 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8b2b6c-19db-4ee1-b328-908e9ee895ee admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:23.092312 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792f1fbf-455b-42cd-8cc5-0c4bd02a030d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:23.092980 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f1fbf-455b-42cd-8cc5-0c4bd02a030d admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:23.093257 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f1fbf-455b-42cd-8cc5-0c4bd02a030d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:23.093529 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792f1fbf-455b-42cd-8cc5-0c4bd02a030d admin admin] PATH_INFO: `/v3/projects/eeb5dbae475a486a9275519f493e496a/users/6c59cd5bcbe8488eb849f450f6023e1e/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:23.096449 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792f1fbf-455b-42cd-8cc5-0c4bd02a030d admin admin] RBAC: Authorizing `identity:create_grant(project_id=eeb5dbae475a486a9275519f493e496a, user_id=6c59cd5bcbe8488eb849f450f6023e1e, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:23.099132 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792f1fbf-455b-42cd-8cc5-0c4bd02a030d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:23.112619 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 798/1606] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:34:23 2022] PUT /identity/v3/projects/eeb5dbae475a486a9275519f493e496a/users/6c59cd5bcbe8488eb849f450f6023e1e/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:34:23.124445 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09276552-dba3-4841-9466-422d30eba4c6 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:23.124923 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09276552-dba3-4841-9466-422d30eba4c6 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:23.126077 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09276552-dba3-4841-9466-422d30eba4c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:23.154671 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-09276552-dba3-4841-9466-422d30eba4c6 None None] MFA Rules not processed for user `6c59cd5bcbe8488eb849f450f6023e1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:34:23.155990 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-09276552-dba3-4841-9466-422d30eba4c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:23.225252 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 809/1607] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:34:23 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 03:34:23.237860 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792f1fbf-455b-42cd-8cc5-0c4bd02a030d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:23.239564 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-792f1fbf-455b-42cd-8cc5-0c4bd02a030d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:23.261563 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:34:23.318905 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792f1fbf-455b-42cd-8cc5-0c4bd02a030d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:23.323314 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7bd06a-422a-4fcd-8c9c-be4e91d33db0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:23.324767 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7bd06a-422a-4fcd-8c9c-be4e91d33db0 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:23.324767 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7bd06a-422a-4fcd-8c9c-be4e91d33db0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:23.325138 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7bd06a-422a-4fcd-8c9c-be4e91d33db0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:23.337384 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7bd06a-422a-4fcd-8c9c-be4e91d33db0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:23.338826 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7bd06a-422a-4fcd-8c9c-be4e91d33db0 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:23.352298 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 799/1608] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:34:23 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:34:23.842700 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09276552-dba3-4841-9466-422d30eba4c6 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:23.858178 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09276552-dba3-4841-9466-422d30eba4c6 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:23.870398 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d894f453-4848-41e8-9035-bbb167b17667 tempest-VolumeQuotaClassesTest-92229097 tempest-VolumeQuotaClassesTest-92229097-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c59cd5bcbe8488eb849f450f6023e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb5dbae475a486a9275519f493e496a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:23.871059 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d894f453-4848-41e8-9035-bbb167b17667 tempest-VolumeQuotaClassesTest-92229097 tempest-VolumeQuotaClassesTest-92229097-project-admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:23.871344 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d894f453-4848-41e8-9035-bbb167b17667 tempest-VolumeQuotaClassesTest-92229097 tempest-VolumeQuotaClassesTest-92229097-project-admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:23.871620 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d894f453-4848-41e8-9035-bbb167b17667 tempest-VolumeQuotaClassesTest-92229097 tempest-VolumeQuotaClassesTest-92229097-project-admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:23.872327 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d894f453-4848-41e8-9035-bbb167b17667 tempest-VolumeQuotaClassesTest-92229097 tempest-VolumeQuotaClassesTest-92229097-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:34:23.872638 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d894f453-4848-41e8-9035-bbb167b17667 tempest-VolumeQuotaClassesTest-92229097 tempest-VolumeQuotaClassesTest-92229097-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:23.874589 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d894f453-4848-41e8-9035-bbb167b17667 tempest-VolumeQuotaClassesTest-92229097 tempest-VolumeQuotaClassesTest-92229097-project-admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:23.895754 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 810/1609] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 03:34:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:34:23.900288 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7bd06a-422a-4fcd-8c9c-be4e91d33db0 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:23.914327 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7bd06a-422a-4fcd-8c9c-be4e91d33db0 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:23.926324 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f0a624-2391-4f59-91e8-a63c81cc1c76 tempest-VolumeQuotaClassesTest-92229097 tempest-VolumeQuotaClassesTest-92229097-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c59cd5bcbe8488eb849f450f6023e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb5dbae475a486a9275519f493e496a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:23.927309 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f0a624-2391-4f59-91e8-a63c81cc1c76 tempest-VolumeQuotaClassesTest-92229097 tempest-VolumeQuotaClassesTest-92229097-project-admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:23.927806 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f0a624-2391-4f59-91e8-a63c81cc1c76 tempest-VolumeQuotaClassesTest-92229097 tempest-VolumeQuotaClassesTest-92229097-project-admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:23.928281 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f0a624-2391-4f59-91e8-a63c81cc1c76 tempest-VolumeQuotaClassesTest-92229097 tempest-VolumeQuotaClassesTest-92229097-project-admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:23.929030 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f0a624-2391-4f59-91e8-a63c81cc1c76 tempest-VolumeQuotaClassesTest-92229097 tempest-VolumeQuotaClassesTest-92229097-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:23.931287 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f0a624-2391-4f59-91e8-a63c81cc1c76 tempest-VolumeQuotaClassesTest-92229097 tempest-VolumeQuotaClassesTest-92229097-project-admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:23.981673 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 800/1610] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:34:23 2022] POST /identity/v3/projects => generated 353 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:34:23.985483 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d894f453-4848-41e8-9035-bbb167b17667 tempest-VolumeQuotaClassesTest-92229097 tempest-VolumeQuotaClassesTest-92229097-project-admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:24.000003 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d894f453-4848-41e8-9035-bbb167b17667 tempest-VolumeQuotaClassesTest-92229097 tempest-VolumeQuotaClassesTest-92229097-project-admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:24.030557 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:34:24.116381 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f80e289-8d5b-4861-b6d4-86c8a675c7ab tempest-VolumeQuotaClassesTest-92229097 tempest-VolumeQuotaClassesTest-92229097-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c59cd5bcbe8488eb849f450f6023e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb5dbae475a486a9275519f493e496a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:24.117286 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f80e289-8d5b-4861-b6d4-86c8a675c7ab tempest-VolumeQuotaClassesTest-92229097 tempest-VolumeQuotaClassesTest-92229097-project-admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:24.117920 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f80e289-8d5b-4861-b6d4-86c8a675c7ab tempest-VolumeQuotaClassesTest-92229097 tempest-VolumeQuotaClassesTest-92229097-project-admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:24.124553 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f80e289-8d5b-4861-b6d4-86c8a675c7ab tempest-VolumeQuotaClassesTest-92229097 tempest-VolumeQuotaClassesTest-92229097-project-admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:24.132044 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f80e289-8d5b-4861-b6d4-86c8a675c7ab tempest-VolumeQuotaClassesTest-92229097 tempest-VolumeQuotaClassesTest-92229097-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:34:24.132044 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f80e289-8d5b-4861-b6d4-86c8a675c7ab tempest-VolumeQuotaClassesTest-92229097 tempest-VolumeQuotaClassesTest-92229097-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:24.135572 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f80e289-8d5b-4861-b6d4-86c8a675c7ab tempest-VolumeQuotaClassesTest-92229097 tempest-VolumeQuotaClassesTest-92229097-project-admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:24.152022 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 811/1611] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 03:34:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:34:24.290106 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f0a624-2391-4f59-91e8-a63c81cc1c76 tempest-VolumeQuotaClassesTest-92229097 tempest-VolumeQuotaClassesTest-92229097-project-admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:24.305945 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f0a624-2391-4f59-91e8-a63c81cc1c76 tempest-VolumeQuotaClassesTest-92229097 tempest-VolumeQuotaClassesTest-92229097-project-admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:24.320963 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02594638-b995-491d-af8d-ff3e07b320a7 tempest-VolumeQuotaClassesTest-92229097 tempest-VolumeQuotaClassesTest-92229097-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6c59cd5bcbe8488eb849f450f6023e1e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'eeb5dbae475a486a9275519f493e496a', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:24.321383 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02594638-b995-491d-af8d-ff3e07b320a7 tempest-VolumeQuotaClassesTest-92229097 tempest-VolumeQuotaClassesTest-92229097-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:24.321759 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02594638-b995-491d-af8d-ff3e07b320a7 tempest-VolumeQuotaClassesTest-92229097 tempest-VolumeQuotaClassesTest-92229097-project-admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:24.322236 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02594638-b995-491d-af8d-ff3e07b320a7 tempest-VolumeQuotaClassesTest-92229097 tempest-VolumeQuotaClassesTest-92229097-project-admin] PATH_INFO: `/v3/projects/36fedfba178b4cf7ae338de3890e07ce` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:24.326326 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02594638-b995-491d-af8d-ff3e07b320a7 tempest-VolumeQuotaClassesTest-92229097 tempest-VolumeQuotaClassesTest-92229097-project-admin] RBAC: Authorizing `identity:delete_project(project_id=36fedfba178b4cf7ae338de3890e07ce)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:24.328537 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02594638-b995-491d-af8d-ff3e07b320a7 tempest-VolumeQuotaClassesTest-92229097 tempest-VolumeQuotaClassesTest-92229097-project-admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:24.434490 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-02594638-b995-491d-af8d-ff3e07b320a7 tempest-VolumeQuotaClassesTest-92229097 tempest-VolumeQuotaClassesTest-92229097-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36fedfba178b4cf7ae338de3890e07ce', 'request_id': 'req-02594638-b995-491d-af8d-ff3e07b320a7'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:24.459714 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-02594638-b995-491d-af8d-ff3e07b320a7 tempest-VolumeQuotaClassesTest-92229097 tempest-VolumeQuotaClassesTest-92229097-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36fedfba178b4cf7ae338de3890e07ce', 'request_id': 'req-02594638-b995-491d-af8d-ff3e07b320a7'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:24.472037 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-02594638-b995-491d-af8d-ff3e07b320a7 tempest-VolumeQuotaClassesTest-92229097 tempest-VolumeQuotaClassesTest-92229097-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36fedfba178b4cf7ae338de3890e07ce', 'request_id': 'req-02594638-b995-491d-af8d-ff3e07b320a7'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:24.485346 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-02594638-b995-491d-af8d-ff3e07b320a7 tempest-VolumeQuotaClassesTest-92229097 tempest-VolumeQuotaClassesTest-92229097-project-admin] The token cache is being invalidate because project 36fedfba178b4cf7ae338de3890e07ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 03:34:24.485918 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-02594638-b995-491d-af8d-ff3e07b320a7 tempest-VolumeQuotaClassesTest-92229097 tempest-VolumeQuotaClassesTest-92229097-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:24.486972 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-02594638-b995-491d-af8d-ff3e07b320a7 tempest-VolumeQuotaClassesTest-92229097 tempest-VolumeQuotaClassesTest-92229097-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:24.488749 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 801/1612] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 03:34:24 2022] DELETE /identity/v3/projects/36fedfba178b4cf7ae338de3890e07ce => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:34:24.760001 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f80e289-8d5b-4861-b6d4-86c8a675c7ab tempest-VolumeQuotaClassesTest-92229097 tempest-VolumeQuotaClassesTest-92229097-project-admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:24.762848 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-0f80e289-8d5b-4861-b6d4-86c8a675c7ab tempest-VolumeQuotaClassesTest-92229097 tempest-VolumeQuotaClassesTest-92229097-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:24.893352 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f80e289-8d5b-4861-b6d4-86c8a675c7ab tempest-VolumeQuotaClassesTest-92229097 tempest-VolumeQuotaClassesTest-92229097-project-admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:24.963643 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652a19fe-db2a-448e-aa60-df0f6fe7c7bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:24.963915 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a19fe-db2a-448e-aa60-df0f6fe7c7bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:24.964185 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a19fe-db2a-448e-aa60-df0f6fe7c7bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:24.964463 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a19fe-db2a-448e-aa60-df0f6fe7c7bf admin admin] PATH_INFO: `/v3/users/08559931aa4546839ab7f2abe38e959d` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:24.965649 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652a19fe-db2a-448e-aa60-df0f6fe7c7bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=08559931aa4546839ab7f2abe38e959d)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:24.967581 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652a19fe-db2a-448e-aa60-df0f6fe7c7bf admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:25.022864 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-652a19fe-db2a-448e-aa60-df0f6fe7c7bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08559931aa4546839ab7f2abe38e959d'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:25.023568 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-652a19fe-db2a-448e-aa60-df0f6fe7c7bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08559931aa4546839ab7f2abe38e959d'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:25.024212 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-652a19fe-db2a-448e-aa60-df0f6fe7c7bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08559931aa4546839ab7f2abe38e959d'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:25.037724 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-652a19fe-db2a-448e-aa60-df0f6fe7c7bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08559931aa4546839ab7f2abe38e959d'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:25.040968 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:34:25.047459 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-652a19fe-db2a-448e-aa60-df0f6fe7c7bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08559931aa4546839ab7f2abe38e959d'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:25.064110 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 812/1613] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 03:34:24 2022] DELETE /identity/v3/users/08559931aa4546839ab7f2abe38e959d => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:34:25.069012 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea7aae0-2dc0-41c6-ae76-50c99c907e3a None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:25.069612 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea7aae0-2dc0-41c6-ae76-50c99c907e3a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:25.070277 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea7aae0-2dc0-41c6-ae76-50c99c907e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:25.094861 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-bea7aae0-2dc0-41c6-ae76-50c99c907e3a None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:34:25.096960 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-bea7aae0-2dc0-41c6-ae76-50c99c907e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:25.182368 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 802/1614] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:34:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:34:25.192131 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652a19fe-db2a-448e-aa60-df0f6fe7c7bf admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:25.198784 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-652a19fe-db2a-448e-aa60-df0f6fe7c7bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:25.318712 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652a19fe-db2a-448e-aa60-df0f6fe7c7bf admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:25.332970 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eda917-d180-47f9-a96a-fb036818c76d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:25.333858 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eda917-d180-47f9-a96a-fb036818c76d service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:25.335438 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eda917-d180-47f9-a96a-fb036818c76d service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:25.335438 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eda917-d180-47f9-a96a-fb036818c76d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:25.351598 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eda917-d180-47f9-a96a-fb036818c76d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:25.353429 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19eda917-d180-47f9-a96a-fb036818c76d service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:25.373548 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 813/1615] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:34:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:34:26.146240 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea7aae0-2dc0-41c6-ae76-50c99c907e3a None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:26.147741 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-bea7aae0-2dc0-41c6-ae76-50c99c907e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:26.160208 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea7aae0-2dc0-41c6-ae76-50c99c907e3a None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:26.165508 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b82498-10b9-46f6-b034-c2d00e84dd45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:26.166233 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b82498-10b9-46f6-b034-c2d00e84dd45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:26.166583 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b82498-10b9-46f6-b034-c2d00e84dd45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:26.166859 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b82498-10b9-46f6-b034-c2d00e84dd45 admin admin] PATH_INFO: `/v3/projects/933bd2bfa38f4dd1aa7dcec2243d2a87` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:26.168476 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b82498-10b9-46f6-b034-c2d00e84dd45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=933bd2bfa38f4dd1aa7dcec2243d2a87)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:26.170599 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b82498-10b9-46f6-b034-c2d00e84dd45 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:26.195414 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:34:26.264765 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d0b82498-10b9-46f6-b034-c2d00e84dd45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '933bd2bfa38f4dd1aa7dcec2243d2a87', 'request_id': 'req-d0b82498-10b9-46f6-b034-c2d00e84dd45'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:26.274984 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d0b82498-10b9-46f6-b034-c2d00e84dd45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '933bd2bfa38f4dd1aa7dcec2243d2a87', 'request_id': 'req-d0b82498-10b9-46f6-b034-c2d00e84dd45'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:26.286243 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d0b82498-10b9-46f6-b034-c2d00e84dd45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '933bd2bfa38f4dd1aa7dcec2243d2a87', 'request_id': 'req-d0b82498-10b9-46f6-b034-c2d00e84dd45'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:26.300839 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d0b82498-10b9-46f6-b034-c2d00e84dd45 admin admin] The token cache is being invalidate because project 933bd2bfa38f4dd1aa7dcec2243d2a87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 03:34:26.301137 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d0b82498-10b9-46f6-b034-c2d00e84dd45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:26.301806 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d0b82498-10b9-46f6-b034-c2d00e84dd45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:26.303316 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 803/1616] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 03:34:26 2022] DELETE /identity/v3/projects/933bd2bfa38f4dd1aa7dcec2243d2a87 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:34:26.307363 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eda917-d180-47f9-a96a-fb036818c76d service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:26.317924 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19eda917-d180-47f9-a96a-fb036818c76d service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:26.410351 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866fb66a-a1a9-4b18-a50d-bfa3257853d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:26.411267 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866fb66a-a1a9-4b18-a50d-bfa3257853d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:26.411838 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866fb66a-a1a9-4b18-a50d-bfa3257853d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:26.412245 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866fb66a-a1a9-4b18-a50d-bfa3257853d8 admin admin] PATH_INFO: `/v3/users/6c59cd5bcbe8488eb849f450f6023e1e` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:26.413524 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866fb66a-a1a9-4b18-a50d-bfa3257853d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c59cd5bcbe8488eb849f450f6023e1e)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:26.415935 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866fb66a-a1a9-4b18-a50d-bfa3257853d8 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:26.517775 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-866fb66a-a1a9-4b18-a50d-bfa3257853d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c59cd5bcbe8488eb849f450f6023e1e'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:26.518517 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-866fb66a-a1a9-4b18-a50d-bfa3257853d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c59cd5bcbe8488eb849f450f6023e1e'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:26.519615 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-866fb66a-a1a9-4b18-a50d-bfa3257853d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c59cd5bcbe8488eb849f450f6023e1e'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:26.546159 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-866fb66a-a1a9-4b18-a50d-bfa3257853d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c59cd5bcbe8488eb849f450f6023e1e'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:26.555435 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-866fb66a-a1a9-4b18-a50d-bfa3257853d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c59cd5bcbe8488eb849f450f6023e1e'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:26.572921 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 814/1617] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 03:34:26 2022] DELETE /identity/v3/users/6c59cd5bcbe8488eb849f450f6023e1e => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:34:27.204944 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b82498-10b9-46f6-b034-c2d00e84dd45 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:27.207022 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d0b82498-10b9-46f6-b034-c2d00e84dd45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:27.220928 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b82498-10b9-46f6-b034-c2d00e84dd45 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:27.294513 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1377830-86ee-40a8-9e5b-db1b6aad14da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:27.295158 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1377830-86ee-40a8-9e5b-db1b6aad14da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:27.295444 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1377830-86ee-40a8-9e5b-db1b6aad14da admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:27.295719 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1377830-86ee-40a8-9e5b-db1b6aad14da admin admin] PATH_INFO: `/v3/projects/eeb5dbae475a486a9275519f493e496a` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:27.296916 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1377830-86ee-40a8-9e5b-db1b6aad14da admin admin] RBAC: Authorizing `identity:delete_project(project_id=eeb5dbae475a486a9275519f493e496a)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:27.299076 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1377830-86ee-40a8-9e5b-db1b6aad14da admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:27.401403 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a1377830-86ee-40a8-9e5b-db1b6aad14da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eeb5dbae475a486a9275519f493e496a', 'request_id': 'req-a1377830-86ee-40a8-9e5b-db1b6aad14da'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:27.415404 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a1377830-86ee-40a8-9e5b-db1b6aad14da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eeb5dbae475a486a9275519f493e496a', 'request_id': 'req-a1377830-86ee-40a8-9e5b-db1b6aad14da'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:27.431431 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a1377830-86ee-40a8-9e5b-db1b6aad14da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eeb5dbae475a486a9275519f493e496a', 'request_id': 'req-a1377830-86ee-40a8-9e5b-db1b6aad14da'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:27.448745 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a1377830-86ee-40a8-9e5b-db1b6aad14da admin admin] The token cache is being invalidate because project eeb5dbae475a486a9275519f493e496a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 03:34:27.449095 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a1377830-86ee-40a8-9e5b-db1b6aad14da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:27.450906 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a1377830-86ee-40a8-9e5b-db1b6aad14da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:27.457089 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 804/1618] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 03:34:27 2022] DELETE /identity/v3/projects/eeb5dbae475a486a9275519f493e496a => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:34:27.458862 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e4fed9-ac45-4a05-949f-449ac0a10bb0 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:27.459303 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e4fed9-ac45-4a05-949f-449ac0a10bb0 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:27.459429 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e4fed9-ac45-4a05-949f-449ac0a10bb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:27.483910 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:34:27.500966 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-56e4fed9-ac45-4a05-949f-449ac0a10bb0 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:34:27.502544 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-56e4fed9-ac45-4a05-949f-449ac0a10bb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:27.606702 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 815/1619] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:34:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:34:27.615096 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febfbafc-0ff1-44b6-9470-3591c9bdabc6 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:27.615388 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febfbafc-0ff1-44b6-9470-3591c9bdabc6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:27.615701 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febfbafc-0ff1-44b6-9470-3591c9bdabc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:27.640329 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-febfbafc-0ff1-44b6-9470-3591c9bdabc6 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:34:27.642450 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-febfbafc-0ff1-44b6-9470-3591c9bdabc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:27.653811 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 805/1620] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:34:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:34:27.659019 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e4fed9-ac45-4a05-949f-449ac0a10bb0 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:27.669694 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e4fed9-ac45-4a05-949f-449ac0a10bb0 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:27.676787 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ce9a05-8375-4f66-bd01-a4656aa8b246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:27.677520 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce9a05-8375-4f66-bd01-a4656aa8b246 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:27.678034 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce9a05-8375-4f66-bd01-a4656aa8b246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:27.678383 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce9a05-8375-4f66-bd01-a4656aa8b246 admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:27.679125 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ce9a05-8375-4f66-bd01-a4656aa8b246 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:34:27.679497 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ce9a05-8375-4f66-bd01-a4656aa8b246 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:27.681223 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ce9a05-8375-4f66-bd01-a4656aa8b246 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:27.697666 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 816/1621] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 03:34:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:34:27.702273 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febfbafc-0ff1-44b6-9470-3591c9bdabc6 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:27.713250 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febfbafc-0ff1-44b6-9470-3591c9bdabc6 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:27.718274 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a819d7fe-27fa-4d77-a953-89af1731f691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:27.718926 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a819d7fe-27fa-4d77-a953-89af1731f691 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:27.719244 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a819d7fe-27fa-4d77-a953-89af1731f691 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:27.719521 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a819d7fe-27fa-4d77-a953-89af1731f691 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:27.720210 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a819d7fe-27fa-4d77-a953-89af1731f691 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:27.722376 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a819d7fe-27fa-4d77-a953-89af1731f691 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:27.761457 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 806/1622] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:34:27 2022] POST /identity/v3/projects => generated 383 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:34:27.763178 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ce9a05-8375-4f66-bd01-a4656aa8b246 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:27.774539 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ce9a05-8375-4f66-bd01-a4656aa8b246 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:27.835130 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528a7f08-986b-4d34-aee6-24843d13b125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:27.835907 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a7f08-986b-4d34-aee6-24843d13b125 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:27.836233 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a7f08-986b-4d34-aee6-24843d13b125 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:27.836514 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a7f08-986b-4d34-aee6-24843d13b125 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:27.837156 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528a7f08-986b-4d34-aee6-24843d13b125 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:27.839813 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528a7f08-986b-4d34-aee6-24843d13b125 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:27.872819 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 817/1623] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:34:27 2022] POST /identity/v3/users => generated 292 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:34:27.876928 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a819d7fe-27fa-4d77-a953-89af1731f691 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:27.888800 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a819d7fe-27fa-4d77-a953-89af1731f691 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:27.893699 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d16eed3-985f-43bb-b6f5-6eecfa5d461a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:27.894326 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16eed3-985f-43bb-b6f5-6eecfa5d461a admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:27.894605 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16eed3-985f-43bb-b6f5-6eecfa5d461a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:27.894888 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16eed3-985f-43bb-b6f5-6eecfa5d461a admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:27.895528 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d16eed3-985f-43bb-b6f5-6eecfa5d461a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:27.897350 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d16eed3-985f-43bb-b6f5-6eecfa5d461a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:27.899312 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:34:27.911434 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 807/1624] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:34:27 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 03:34:27.914880 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528a7f08-986b-4d34-aee6-24843d13b125 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:27.925411 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528a7f08-986b-4d34-aee6-24843d13b125 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:27.930203 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacc8d78-0fe5-42cb-8074-aa62b693c946 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:27.930813 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc8d78-0fe5-42cb-8074-aa62b693c946 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:27.931094 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc8d78-0fe5-42cb-8074-aa62b693c946 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:27.931369 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacc8d78-0fe5-42cb-8074-aa62b693c946 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:27.932068 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacc8d78-0fe5-42cb-8074-aa62b693c946 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:27.933651 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacc8d78-0fe5-42cb-8074-aa62b693c946 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:27.944336 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 818/1625] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:34:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:34:27.948179 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d16eed3-985f-43bb-b6f5-6eecfa5d461a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:27.958299 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d16eed3-985f-43bb-b6f5-6eecfa5d461a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:27.962946 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e7e3e2-5e9f-499f-90d3-ec1f27a73619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:27.963587 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7e3e2-5e9f-499f-90d3-ec1f27a73619 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:27.963867 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7e3e2-5e9f-499f-90d3-ec1f27a73619 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:27.964198 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e7e3e2-5e9f-499f-90d3-ec1f27a73619 admin admin] PATH_INFO: `/v3/projects/4fd0d4a99b284782baacf69032f036ad/users/6f43ba1b3696426db25023878b21681b/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:27.979440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e7e3e2-5e9f-499f-90d3-ec1f27a73619 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fd0d4a99b284782baacf69032f036ad, user_id=6f43ba1b3696426db25023878b21681b, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:27.982099 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e7e3e2-5e9f-499f-90d3-ec1f27a73619 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:27.995757 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 808/1626] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:34:27 2022] PUT /identity/v3/projects/4fd0d4a99b284782baacf69032f036ad/users/6f43ba1b3696426db25023878b21681b/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:34:28.006358 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec3a3a7-8e1a-497a-ab72-c5a289d04491 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:28.006643 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec3a3a7-8e1a-497a-ab72-c5a289d04491 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:28.006909 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec3a3a7-8e1a-497a-ab72-c5a289d04491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:28.034057 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-aec3a3a7-8e1a-497a-ab72-c5a289d04491 None None] MFA Rules not processed for user `6f43ba1b3696426db25023878b21681b`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:34:28.035240 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-aec3a3a7-8e1a-497a-ab72-c5a289d04491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:28.099509 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 819/1627] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:34:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:34:28.104022 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e7e3e2-5e9f-499f-90d3-ec1f27a73619 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:28.170797 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e7e3e2-5e9f-499f-90d3-ec1f27a73619 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:28.175933 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11f29b9-a4ac-465d-a128-789b699b539c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:28.176578 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f29b9-a4ac-465d-a128-789b699b539c admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:28.176855 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f29b9-a4ac-465d-a128-789b699b539c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:28.177200 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f29b9-a4ac-465d-a128-789b699b539c admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:28.177874 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11f29b9-a4ac-465d-a128-789b699b539c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:28.179772 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11f29b9-a4ac-465d-a128-789b699b539c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:28.206137 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 809/1628] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:34:28 2022] POST /identity/v3/projects => generated 381 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:34:28.210259 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec3a3a7-8e1a-497a-ab72-c5a289d04491 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:28.215617 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:34:28.223421 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec3a3a7-8e1a-497a-ab72-c5a289d04491 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:28.281420 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109beee8-e4a0-466b-af26-fc76cdcf7cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:28.282577 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109beee8-e4a0-466b-af26-fc76cdcf7cf3 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:28.282856 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109beee8-e4a0-466b-af26-fc76cdcf7cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:28.283128 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109beee8-e4a0-466b-af26-fc76cdcf7cf3 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:28.283789 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109beee8-e4a0-466b-af26-fc76cdcf7cf3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:28.285742 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109beee8-e4a0-466b-af26-fc76cdcf7cf3 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:28.307820 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 820/1629] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:34:28 2022] POST /identity/v3/users => generated 291 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:34:28.311888 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11f29b9-a4ac-465d-a128-789b699b539c admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:28.322565 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11f29b9-a4ac-465d-a128-789b699b539c admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:28.327249 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf90660d-7d5e-4473-af03-57c961c69960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:28.327855 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf90660d-7d5e-4473-af03-57c961c69960 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:28.328168 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf90660d-7d5e-4473-af03-57c961c69960 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:28.328934 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf90660d-7d5e-4473-af03-57c961c69960 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:28.329570 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf90660d-7d5e-4473-af03-57c961c69960 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:28.331167 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf90660d-7d5e-4473-af03-57c961c69960 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:28.341837 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 810/1630] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:34:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 03:34:28.346328 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109beee8-e4a0-466b-af26-fc76cdcf7cf3 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:28.356257 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109beee8-e4a0-466b-af26-fc76cdcf7cf3 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:28.360831 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70ecd31-8e40-4121-8f63-113c55b4c26f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:28.361427 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ecd31-8e40-4121-8f63-113c55b4c26f admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:28.361705 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ecd31-8e40-4121-8f63-113c55b4c26f admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:28.361976 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70ecd31-8e40-4121-8f63-113c55b4c26f admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:28.362608 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70ecd31-8e40-4121-8f63-113c55b4c26f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:28.364234 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70ecd31-8e40-4121-8f63-113c55b4c26f admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:28.374274 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 821/1631] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:34:28 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:34:28.378897 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf90660d-7d5e-4473-af03-57c961c69960 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:28.388657 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf90660d-7d5e-4473-af03-57c961c69960 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:28.393863 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f542dcd-8428-476f-99a8-61bd093319ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:28.394502 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f542dcd-8428-476f-99a8-61bd093319ec admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:28.394776 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f542dcd-8428-476f-99a8-61bd093319ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:28.395045 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f542dcd-8428-476f-99a8-61bd093319ec admin admin] PATH_INFO: `/v3/projects/6afa5af3cfe24220aa662503a7db6728/users/00746ccffbe04f63ae28b5bbb1a303a3/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:28.411050 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f542dcd-8428-476f-99a8-61bd093319ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=6afa5af3cfe24220aa662503a7db6728, user_id=00746ccffbe04f63ae28b5bbb1a303a3, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:28.413869 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f542dcd-8428-476f-99a8-61bd093319ec admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:28.428728 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 811/1632] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:34:28 2022] PUT /identity/v3/projects/6afa5af3cfe24220aa662503a7db6728/users/00746ccffbe04f63ae28b5bbb1a303a3/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:34:28.440722 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d71d13-2866-4f13-b46d-6f84c0f9022b None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:28.441091 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d71d13-2866-4f13-b46d-6f84c0f9022b None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:28.441376 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d71d13-2866-4f13-b46d-6f84c0f9022b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:28.491067 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-e8d71d13-2866-4f13-b46d-6f84c0f9022b None None] MFA Rules not processed for user `00746ccffbe04f63ae28b5bbb1a303a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:34:28.492435 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e8d71d13-2866-4f13-b46d-6f84c0f9022b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:28.611734 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 822/1633] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:34:28 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:34:28.616674 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f542dcd-8428-476f-99a8-61bd093319ec admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:28.691871 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f542dcd-8428-476f-99a8-61bd093319ec admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:28.700254 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c5deab-61a3-42f5-b6ad-2b3ebf7fdeab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:28.700885 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c5deab-61a3-42f5-b6ad-2b3ebf7fdeab admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:28.701175 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c5deab-61a3-42f5-b6ad-2b3ebf7fdeab admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:28.701439 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c5deab-61a3-42f5-b6ad-2b3ebf7fdeab admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:28.702106 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c5deab-61a3-42f5-b6ad-2b3ebf7fdeab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:28.704246 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51c5deab-61a3-42f5-b6ad-2b3ebf7fdeab admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:28.736989 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 812/1634] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:34:28 2022] POST /identity/v3/projects => generated 383 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:34:28.740985 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d71d13-2866-4f13-b46d-6f84c0f9022b None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:28.753051 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d71d13-2866-4f13-b46d-6f84c0f9022b None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:28.838152 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c36f06-447c-4e50-96ae-f7fc3ae25bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:28.838770 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c36f06-447c-4e50-96ae-f7fc3ae25bd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:28.839046 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c36f06-447c-4e50-96ae-f7fc3ae25bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:28.839317 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c36f06-447c-4e50-96ae-f7fc3ae25bd0 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:28.839999 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c36f06-447c-4e50-96ae-f7fc3ae25bd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:28.842040 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c36f06-447c-4e50-96ae-f7fc3ae25bd0 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:28.859059 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:34:28.865579 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 823/1635] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:34:28 2022] POST /identity/v3/users => generated 298 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:34:28.869495 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c5deab-61a3-42f5-b6ad-2b3ebf7fdeab admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:28.880805 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c5deab-61a3-42f5-b6ad-2b3ebf7fdeab admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:28.885707 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa805f9c-beea-4835-b249-95c092b3e3a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:28.886334 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa805f9c-beea-4835-b249-95c092b3e3a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:28.886616 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa805f9c-beea-4835-b249-95c092b3e3a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:28.886897 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa805f9c-beea-4835-b249-95c092b3e3a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:28.887587 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa805f9c-beea-4835-b249-95c092b3e3a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:28.889235 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa805f9c-beea-4835-b249-95c092b3e3a5 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:28.890666 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:34:28.903998 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 813/1636] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:34:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:34:28.906960 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c36f06-447c-4e50-96ae-f7fc3ae25bd0 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:28.917348 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c36f06-447c-4e50-96ae-f7fc3ae25bd0 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:28.922679 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeb9a7d-d11c-4cf1-8603-eefdf1ebc81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:28.923351 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb9a7d-d11c-4cf1-8603-eefdf1ebc81b admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:28.923632 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb9a7d-d11c-4cf1-8603-eefdf1ebc81b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:28.923907 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deeb9a7d-d11c-4cf1-8603-eefdf1ebc81b admin admin] PATH_INFO: `/v3/projects/24d69da55ab84cd19733d48c88e7521e/users/1f392fc998dc4c9995273a8aa4888ada/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:28.941494 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeb9a7d-d11c-4cf1-8603-eefdf1ebc81b admin admin] RBAC: Authorizing `identity:create_grant(project_id=24d69da55ab84cd19733d48c88e7521e, user_id=1f392fc998dc4c9995273a8aa4888ada, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:28.944416 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deeb9a7d-d11c-4cf1-8603-eefdf1ebc81b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:28.959241 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 824/1637] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:34:28 2022] PUT /identity/v3/projects/24d69da55ab84cd19733d48c88e7521e/users/1f392fc998dc4c9995273a8aa4888ada/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:34:28.963710 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa805f9c-beea-4835-b249-95c092b3e3a5 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:28.975567 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa805f9c-beea-4835-b249-95c092b3e3a5 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:29.039486 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfcbe04-e157-410e-8ebe-7c3d6e82a481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:29.040122 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfcbe04-e157-410e-8ebe-7c3d6e82a481 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:29.040403 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfcbe04-e157-410e-8ebe-7c3d6e82a481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:29.040673 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfcbe04-e157-410e-8ebe-7c3d6e82a481 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:29.041296 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfcbe04-e157-410e-8ebe-7c3d6e82a481 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:29.042880 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfcbe04-e157-410e-8ebe-7c3d6e82a481 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:29.053016 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 814/1638] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:34:28 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:34:29.057184 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeb9a7d-d11c-4cf1-8603-eefdf1ebc81b admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:29.068424 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deeb9a7d-d11c-4cf1-8603-eefdf1ebc81b admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:29.073169 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cda102e-3099-4eba-b9e5-41ed11f83588 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:29.073816 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda102e-3099-4eba-b9e5-41ed11f83588 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:29.074089 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda102e-3099-4eba-b9e5-41ed11f83588 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:29.074355 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda102e-3099-4eba-b9e5-41ed11f83588 admin admin] PATH_INFO: `/v3/projects/24d69da55ab84cd19733d48c88e7521e/users/1f392fc998dc4c9995273a8aa4888ada/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:29.076261 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cda102e-3099-4eba-b9e5-41ed11f83588 admin admin] RBAC: Authorizing `identity:create_grant(project_id=24d69da55ab84cd19733d48c88e7521e, user_id=1f392fc998dc4c9995273a8aa4888ada, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:29.078915 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cda102e-3099-4eba-b9e5-41ed11f83588 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:29.089062 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 825/1639] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:34:29 2022] PUT /identity/v3/projects/24d69da55ab84cd19733d48c88e7521e/users/1f392fc998dc4c9995273a8aa4888ada/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:34:29.099670 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1faa6d1-9de2-4907-ac03-fa41c3210785 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:29.099969 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1faa6d1-9de2-4907-ac03-fa41c3210785 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:29.100266 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1faa6d1-9de2-4907-ac03-fa41c3210785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:29.129151 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d1faa6d1-9de2-4907-ac03-fa41c3210785 None None] MFA Rules not processed for user `1f392fc998dc4c9995273a8aa4888ada`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:34:29.130366 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d1faa6d1-9de2-4907-ac03-fa41c3210785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:29.206733 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 815/1640] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:34:29 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:34:29.213939 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cda102e-3099-4eba-b9e5-41ed11f83588 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:29.215782 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-5cda102e-3099-4eba-b9e5-41ed11f83588 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:29.275102 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cda102e-3099-4eba-b9e5-41ed11f83588 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:29.279068 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4093897e-41be-44fd-8e5d-e21f0f03f7ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:29.279744 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093897e-41be-44fd-8e5d-e21f0f03f7ab service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:29.280075 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093897e-41be-44fd-8e5d-e21f0f03f7ab service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:29.280390 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4093897e-41be-44fd-8e5d-e21f0f03f7ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:29.338814 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4093897e-41be-44fd-8e5d-e21f0f03f7ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:29.340289 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4093897e-41be-44fd-8e5d-e21f0f03f7ab service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:29.350881 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 826/1641] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:34:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:34:30.927813 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1faa6d1-9de2-4907-ac03-fa41c3210785 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:30.937394 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1faa6d1-9de2-4907-ac03-fa41c3210785 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:30.941559 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e78a47e-ce88-40d8-9a47-0e77db732985 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:30.942170 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e78a47e-ce88-40d8-9a47-0e77db732985 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:30.942445 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e78a47e-ce88-40d8-9a47-0e77db732985 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:30.942875 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e78a47e-ce88-40d8-9a47-0e77db732985 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:31.007253 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e78a47e-ce88-40d8-9a47-0e77db732985 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:31.009517 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e78a47e-ce88-40d8-9a47-0e77db732985 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:31.025838 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 816/1642] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:34:30 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:34:31.774047 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4093897e-41be-44fd-8e5d-e21f0f03f7ab service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:31.775557 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-4093897e-41be-44fd-8e5d-e21f0f03f7ab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:31.787528 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4093897e-41be-44fd-8e5d-e21f0f03f7ab service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:31.792863 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a98f6d7-bb10-48ff-a908-7989dc0c199d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:31.793688 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a98f6d7-bb10-48ff-a908-7989dc0c199d service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:31.793969 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a98f6d7-bb10-48ff-a908-7989dc0c199d service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:31.794240 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a98f6d7-bb10-48ff-a908-7989dc0c199d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:31.803620 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a98f6d7-bb10-48ff-a908-7989dc0c199d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:31.805557 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a98f6d7-bb10-48ff-a908-7989dc0c199d service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:31.817048 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 827/1643] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:34:31 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:34:35.763170 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e78a47e-ce88-40d8-9a47-0e77db732985 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:35.780060 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:34:35.825237 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e78a47e-ce88-40d8-9a47-0e77db732985 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:35.830402 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a098182-53eb-4bb2-bb8b-9cfb18463aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:35.831033 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a098182-53eb-4bb2-bb8b-9cfb18463aea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:35.831316 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a098182-53eb-4bb2-bb8b-9cfb18463aea admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:35.831597 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a098182-53eb-4bb2-bb8b-9cfb18463aea admin admin] PATH_INFO: `/v3/users/6f43ba1b3696426db25023878b21681b` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:35.832748 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a098182-53eb-4bb2-bb8b-9cfb18463aea admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f43ba1b3696426db25023878b21681b)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:35.834717 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a098182-53eb-4bb2-bb8b-9cfb18463aea admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:35.886323 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8a098182-53eb-4bb2-bb8b-9cfb18463aea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f43ba1b3696426db25023878b21681b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:35.893825 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8a098182-53eb-4bb2-bb8b-9cfb18463aea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f43ba1b3696426db25023878b21681b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:35.894520 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8a098182-53eb-4bb2-bb8b-9cfb18463aea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f43ba1b3696426db25023878b21681b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:35.906456 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8a098182-53eb-4bb2-bb8b-9cfb18463aea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f43ba1b3696426db25023878b21681b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:35.914028 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8a098182-53eb-4bb2-bb8b-9cfb18463aea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f43ba1b3696426db25023878b21681b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:35.920074 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 817/1644] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 03:34:35 2022] DELETE /identity/v3/users/6f43ba1b3696426db25023878b21681b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:34:35.925131 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b10fd-1dae-4f9c-9d4a-027870f12738 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:35.925578 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b10fd-1dae-4f9c-9d4a-027870f12738 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:35.925699 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05b10fd-1dae-4f9c-9d4a-027870f12738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:35.947941 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-f05b10fd-1dae-4f9c-9d4a-027870f12738 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:34:35.949249 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f05b10fd-1dae-4f9c-9d4a-027870f12738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:36.023174 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 828/1645] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:34:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:34:36.033219 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a098182-53eb-4bb2-bb8b-9cfb18463aea admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:36.034726 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8a098182-53eb-4bb2-bb8b-9cfb18463aea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:36.092083 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a098182-53eb-4bb2-bb8b-9cfb18463aea admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:36.096163 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c30911-d0fd-481a-9a60-9c4f1773fad4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:36.096772 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c30911-d0fd-481a-9a60-9c4f1773fad4 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:36.097051 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c30911-d0fd-481a-9a60-9c4f1773fad4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:36.097323 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c30911-d0fd-481a-9a60-9c4f1773fad4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:36.106229 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c30911-d0fd-481a-9a60-9c4f1773fad4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:36.107639 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c30911-d0fd-481a-9a60-9c4f1773fad4 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:36.117674 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 818/1646] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:34:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:34:36.390339 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05b10fd-1dae-4f9c-9d4a-027870f12738 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:36.393500 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f05b10fd-1dae-4f9c-9d4a-027870f12738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:36.417974 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:34:36.425398 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05b10fd-1dae-4f9c-9d4a-027870f12738 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:36.432493 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3807da30-5268-4c86-aca5-960ef8333a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:36.433150 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3807da30-5268-4c86-aca5-960ef8333a94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:36.433455 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3807da30-5268-4c86-aca5-960ef8333a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:36.433723 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3807da30-5268-4c86-aca5-960ef8333a94 admin admin] PATH_INFO: `/v3/projects/4fd0d4a99b284782baacf69032f036ad` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:36.434951 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3807da30-5268-4c86-aca5-960ef8333a94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fd0d4a99b284782baacf69032f036ad)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:36.436924 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3807da30-5268-4c86-aca5-960ef8333a94 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:36.484061 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3807da30-5268-4c86-aca5-960ef8333a94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fd0d4a99b284782baacf69032f036ad', 'request_id': 'req-3807da30-5268-4c86-aca5-960ef8333a94'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:36.489644 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3807da30-5268-4c86-aca5-960ef8333a94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fd0d4a99b284782baacf69032f036ad', 'request_id': 'req-3807da30-5268-4c86-aca5-960ef8333a94'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:36.495722 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3807da30-5268-4c86-aca5-960ef8333a94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fd0d4a99b284782baacf69032f036ad', 'request_id': 'req-3807da30-5268-4c86-aca5-960ef8333a94'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:36.508718 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3807da30-5268-4c86-aca5-960ef8333a94 admin admin] The token cache is being invalidate because project 4fd0d4a99b284782baacf69032f036ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 03:34:36.509007 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3807da30-5268-4c86-aca5-960ef8333a94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:36.509627 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3807da30-5268-4c86-aca5-960ef8333a94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:36.511192 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 829/1647] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 03:34:36 2022] DELETE /identity/v3/projects/4fd0d4a99b284782baacf69032f036ad => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:34:36.515196 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c30911-d0fd-481a-9a60-9c4f1773fad4 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:36.525473 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c30911-d0fd-481a-9a60-9c4f1773fad4 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:36.559670 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:34:36.584537 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af6257d-633b-469f-959d-7ec7c4bd6591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:36.585172 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af6257d-633b-469f-959d-7ec7c4bd6591 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:36.585455 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af6257d-633b-469f-959d-7ec7c4bd6591 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:36.585734 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af6257d-633b-469f-959d-7ec7c4bd6591 admin admin] PATH_INFO: `/v3/users/00746ccffbe04f63ae28b5bbb1a303a3` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:36.586856 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af6257d-633b-469f-959d-7ec7c4bd6591 admin admin] RBAC: Authorizing `identity:delete_user(user_id=00746ccffbe04f63ae28b5bbb1a303a3)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:36.588859 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af6257d-633b-469f-959d-7ec7c4bd6591 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:36.634367 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-3af6257d-633b-469f-959d-7ec7c4bd6591 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00746ccffbe04f63ae28b5bbb1a303a3'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:36.641664 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-3af6257d-633b-469f-959d-7ec7c4bd6591 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00746ccffbe04f63ae28b5bbb1a303a3'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:36.642321 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-3af6257d-633b-469f-959d-7ec7c4bd6591 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00746ccffbe04f63ae28b5bbb1a303a3'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:36.653878 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-3af6257d-633b-469f-959d-7ec7c4bd6591 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00746ccffbe04f63ae28b5bbb1a303a3'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:36.661670 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-3af6257d-633b-469f-959d-7ec7c4bd6591 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00746ccffbe04f63ae28b5bbb1a303a3'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:36.667549 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 819/1648] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 03:34:36 2022] DELETE /identity/v3/users/00746ccffbe04f63ae28b5bbb1a303a3 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:34:36.940130 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3807da30-5268-4c86-aca5-960ef8333a94 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:36.941569 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-3807da30-5268-4c86-aca5-960ef8333a94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:36.957306 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3807da30-5268-4c86-aca5-960ef8333a94 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:37.016401 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df019af-7936-48ff-a157-d33d6a83e8fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:37.017048 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df019af-7936-48ff-a157-d33d6a83e8fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:37.017328 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df019af-7936-48ff-a157-d33d6a83e8fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:37.017597 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df019af-7936-48ff-a157-d33d6a83e8fc admin admin] PATH_INFO: `/v3/projects/6afa5af3cfe24220aa662503a7db6728` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:37.018725 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df019af-7936-48ff-a157-d33d6a83e8fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=6afa5af3cfe24220aa662503a7db6728)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:37.020688 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df019af-7936-48ff-a157-d33d6a83e8fc admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:37.069413 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2df019af-7936-48ff-a157-d33d6a83e8fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6afa5af3cfe24220aa662503a7db6728', 'request_id': 'req-2df019af-7936-48ff-a157-d33d6a83e8fc'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:37.076569 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2df019af-7936-48ff-a157-d33d6a83e8fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6afa5af3cfe24220aa662503a7db6728', 'request_id': 'req-2df019af-7936-48ff-a157-d33d6a83e8fc'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:37.088608 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2df019af-7936-48ff-a157-d33d6a83e8fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6afa5af3cfe24220aa662503a7db6728', 'request_id': 'req-2df019af-7936-48ff-a157-d33d6a83e8fc'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:37.102228 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2df019af-7936-48ff-a157-d33d6a83e8fc admin admin] The token cache is being invalidate because project 6afa5af3cfe24220aa662503a7db6728 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 03:34:37.102511 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2df019af-7936-48ff-a157-d33d6a83e8fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:37.103135 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2df019af-7936-48ff-a157-d33d6a83e8fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:37.104680 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 830/1649] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 03:34:36 2022] DELETE /identity/v3/projects/6afa5af3cfe24220aa662503a7db6728 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 03:34:37.109444 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af6257d-633b-469f-959d-7ec7c4bd6591 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:37.111134 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3af6257d-633b-469f-959d-7ec7c4bd6591 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:37.124722 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af6257d-633b-469f-959d-7ec7c4bd6591 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:37.204423 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d059d9-fc7b-4946-abc0-fb643c688e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:37.205088 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d059d9-fc7b-4946-abc0-fb643c688e41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:37.205360 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d059d9-fc7b-4946-abc0-fb643c688e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:37.205628 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d059d9-fc7b-4946-abc0-fb643c688e41 admin admin] PATH_INFO: `/v3/users/1f392fc998dc4c9995273a8aa4888ada` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:37.206725 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d059d9-fc7b-4946-abc0-fb643c688e41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f392fc998dc4c9995273a8aa4888ada)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:37.208670 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d059d9-fc7b-4946-abc0-fb643c688e41 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:37.256142 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e7d059d9-fc7b-4946-abc0-fb643c688e41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f392fc998dc4c9995273a8aa4888ada'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:37.263026 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e7d059d9-fc7b-4946-abc0-fb643c688e41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f392fc998dc4c9995273a8aa4888ada'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:37.263691 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e7d059d9-fc7b-4946-abc0-fb643c688e41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f392fc998dc4c9995273a8aa4888ada'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:37.265572 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:34:37.277743 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e7d059d9-fc7b-4946-abc0-fb643c688e41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f392fc998dc4c9995273a8aa4888ada'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:37.292307 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e7d059d9-fc7b-4946-abc0-fb643c688e41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f392fc998dc4c9995273a8aa4888ada'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:37.298415 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 820/1650] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 03:34:37 2022] DELETE /identity/v3/users/1f392fc998dc4c9995273a8aa4888ada => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:34:37.561881 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df019af-7936-48ff-a157-d33d6a83e8fc admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:37.564032 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-2df019af-7936-48ff-a157-d33d6a83e8fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:37.577681 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df019af-7936-48ff-a157-d33d6a83e8fc admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:37.590899 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:34:37.636409 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38a44db-82f0-4b73-9a3b-40a8fb308a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:37.637062 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a44db-82f0-4b73-9a3b-40a8fb308a5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:37.637345 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a44db-82f0-4b73-9a3b-40a8fb308a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:37.637620 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a44db-82f0-4b73-9a3b-40a8fb308a5d admin admin] PATH_INFO: `/v3/projects/24d69da55ab84cd19733d48c88e7521e` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:37.638732 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38a44db-82f0-4b73-9a3b-40a8fb308a5d admin admin] RBAC: Authorizing `identity:delete_project(project_id=24d69da55ab84cd19733d48c88e7521e)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:37.640707 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38a44db-82f0-4b73-9a3b-40a8fb308a5d admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:37.686685 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b38a44db-82f0-4b73-9a3b-40a8fb308a5d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '24d69da55ab84cd19733d48c88e7521e', 'request_id': 'req-b38a44db-82f0-4b73-9a3b-40a8fb308a5d'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:37.692891 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b38a44db-82f0-4b73-9a3b-40a8fb308a5d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '24d69da55ab84cd19733d48c88e7521e', 'request_id': 'req-b38a44db-82f0-4b73-9a3b-40a8fb308a5d'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:37.699085 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b38a44db-82f0-4b73-9a3b-40a8fb308a5d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '24d69da55ab84cd19733d48c88e7521e', 'request_id': 'req-b38a44db-82f0-4b73-9a3b-40a8fb308a5d'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:37.711607 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b38a44db-82f0-4b73-9a3b-40a8fb308a5d admin admin] The token cache is being invalidate because project 24d69da55ab84cd19733d48c88e7521e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 03:34:37.711899 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b38a44db-82f0-4b73-9a3b-40a8fb308a5d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:37.712545 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b38a44db-82f0-4b73-9a3b-40a8fb308a5d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:37.714017 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 831/1651] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 03:34:37 2022] DELETE /identity/v3/projects/24d69da55ab84cd19733d48c88e7521e => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:34:37.719342 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66235dc5-c0b1-4b3f-a0cd-da48106bf6f4 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:37.719653 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66235dc5-c0b1-4b3f-a0cd-da48106bf6f4 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:37.719930 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66235dc5-c0b1-4b3f-a0cd-da48106bf6f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:37.739646 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-66235dc5-c0b1-4b3f-a0cd-da48106bf6f4 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:34:37.740871 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-66235dc5-c0b1-4b3f-a0cd-da48106bf6f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:37.801338 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 821/1652] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:34:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:34:37.810752 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd38efb8-6b7b-40d1-a741-5e5d12c232a4 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:37.811033 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd38efb8-6b7b-40d1-a741-5e5d12c232a4 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:37.811326 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd38efb8-6b7b-40d1-a741-5e5d12c232a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:37.831246 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-fd38efb8-6b7b-40d1-a741-5e5d12c232a4 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:34:37.832487 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-fd38efb8-6b7b-40d1-a741-5e5d12c232a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:37.840468 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 832/1653] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:34:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:34:37.845230 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66235dc5-c0b1-4b3f-a0cd-da48106bf6f4 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:37.854947 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66235dc5-c0b1-4b3f-a0cd-da48106bf6f4 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:37.859546 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38a6ff8-0818-46ad-b802-48e74debd33b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:37.860176 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a6ff8-0818-46ad-b802-48e74debd33b admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:37.860456 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a6ff8-0818-46ad-b802-48e74debd33b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:37.860756 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a6ff8-0818-46ad-b802-48e74debd33b admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:37.861396 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38a6ff8-0818-46ad-b802-48e74debd33b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:34:37.861696 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38a6ff8-0818-46ad-b802-48e74debd33b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:37.863276 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38a6ff8-0818-46ad-b802-48e74debd33b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:37.874813 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 822/1654] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 03:34:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:34:37.879415 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd38efb8-6b7b-40d1-a741-5e5d12c232a4 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:37.888542 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd38efb8-6b7b-40d1-a741-5e5d12c232a4 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:37.893177 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576fb354-7547-4304-acd9-0da43211ced0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:37.893772 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576fb354-7547-4304-acd9-0da43211ced0 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:37.894050 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576fb354-7547-4304-acd9-0da43211ced0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:37.894321 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576fb354-7547-4304-acd9-0da43211ced0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:37.894945 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576fb354-7547-4304-acd9-0da43211ced0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:37.896858 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576fb354-7547-4304-acd9-0da43211ced0 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:37.925732 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 833/1655] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:34:37 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:34:37.929765 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38a6ff8-0818-46ad-b802-48e74debd33b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:37.939637 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38a6ff8-0818-46ad-b802-48e74debd33b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:37.994264 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d162a1-75c9-444c-80c1-ef51a84e7e14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:37.994862 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d162a1-75c9-444c-80c1-ef51a84e7e14 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:37.995140 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d162a1-75c9-444c-80c1-ef51a84e7e14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:37.995412 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d162a1-75c9-444c-80c1-ef51a84e7e14 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:37.996062 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d162a1-75c9-444c-80c1-ef51a84e7e14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:37.997954 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d162a1-75c9-444c-80c1-ef51a84e7e14 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:38.018733 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 823/1656] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:34:37 2022] POST /identity/v3/users => generated 288 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:34:38.022782 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576fb354-7547-4304-acd9-0da43211ced0 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:38.033389 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576fb354-7547-4304-acd9-0da43211ced0 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:38.037890 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bee009-6598-42bd-97c2-b20d7bb45443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:38.038481 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bee009-6598-42bd-97c2-b20d7bb45443 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:38.038752 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bee009-6598-42bd-97c2-b20d7bb45443 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:38.039015 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bee009-6598-42bd-97c2-b20d7bb45443 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:38.039638 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bee009-6598-42bd-97c2-b20d7bb45443 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:38.041248 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bee009-6598-42bd-97c2-b20d7bb45443 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:38.050924 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 834/1657] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:34:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:34:38.054544 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d162a1-75c9-444c-80c1-ef51a84e7e14 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:38.064389 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d162a1-75c9-444c-80c1-ef51a84e7e14 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:38.068956 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d92d8c-8ca1-4a01-9fcb-1765552f2b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:38.069557 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d92d8c-8ca1-4a01-9fcb-1765552f2b01 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:38.069923 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d92d8c-8ca1-4a01-9fcb-1765552f2b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:38.070166 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d92d8c-8ca1-4a01-9fcb-1765552f2b01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:38.070802 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d92d8c-8ca1-4a01-9fcb-1765552f2b01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:38.072406 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d92d8c-8ca1-4a01-9fcb-1765552f2b01 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:38.083537 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 824/1658] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:34:38 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:34:38.087499 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bee009-6598-42bd-97c2-b20d7bb45443 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:38.097376 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bee009-6598-42bd-97c2-b20d7bb45443 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:38.101872 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fa63d8-3db1-4968-be81-192001fad934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:38.102524 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa63d8-3db1-4968-be81-192001fad934 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:38.102801 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa63d8-3db1-4968-be81-192001fad934 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:38.103073 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fa63d8-3db1-4968-be81-192001fad934 admin admin] PATH_INFO: `/v3/projects/0735003cd30c47298281be972541f8a2/users/a389b51511644982882723092ffddc93/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:38.118635 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fa63d8-3db1-4968-be81-192001fad934 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0735003cd30c47298281be972541f8a2, user_id=a389b51511644982882723092ffddc93, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:38.121300 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fa63d8-3db1-4968-be81-192001fad934 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:38.136622 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 835/1659] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:34:38 2022] PUT /identity/v3/projects/0735003cd30c47298281be972541f8a2/users/a389b51511644982882723092ffddc93/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:34:38.147208 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c959b4-c6c5-45cd-86d4-bde19fb43c53 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:38.147725 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c959b4-c6c5-45cd-86d4-bde19fb43c53 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:38.147910 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c959b4-c6c5-45cd-86d4-bde19fb43c53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:38.181460 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-14c959b4-c6c5-45cd-86d4-bde19fb43c53 None None] MFA Rules not processed for user `a389b51511644982882723092ffddc93`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:34:38.182891 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-14c959b4-c6c5-45cd-86d4-bde19fb43c53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:38.217212 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:34:38.257068 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 825/1660] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:34:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:34:38.267507 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fa63d8-3db1-4968-be81-192001fad934 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:38.269404 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f0fa63d8-3db1-4968-be81-192001fad934 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:38.321054 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:34:38.336082 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fa63d8-3db1-4968-be81-192001fad934 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:38.347583 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6439706-4c39-4288-9fef-35d269553675 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:38.348253 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6439706-4c39-4288-9fef-35d269553675 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:38.348538 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6439706-4c39-4288-9fef-35d269553675 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:38.348845 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6439706-4c39-4288-9fef-35d269553675 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:38.357822 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6439706-4c39-4288-9fef-35d269553675 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:38.359240 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6439706-4c39-4288-9fef-35d269553675 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:38.368105 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 836/1661] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:34:38 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:34:48.252370 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=13 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:34:48.364145 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=6 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:34:48.613959 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c959b4-c6c5-45cd-86d4-bde19fb43c53 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:48.704699 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c959b4-c6c5-45cd-86d4-bde19fb43c53 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:48.711329 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc84eb6-cae6-44a5-8360-e3906564c51b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:48.712254 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc84eb6-cae6-44a5-8360-e3906564c51b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:48.712533 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc84eb6-cae6-44a5-8360-e3906564c51b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:48.712806 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc84eb6-cae6-44a5-8360-e3906564c51b admin admin] PATH_INFO: `/v3/users/a389b51511644982882723092ffddc93` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:48.714230 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc84eb6-cae6-44a5-8360-e3906564c51b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a389b51511644982882723092ffddc93)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:48.716444 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc84eb6-cae6-44a5-8360-e3906564c51b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:48.784782 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cfc84eb6-cae6-44a5-8360-e3906564c51b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a389b51511644982882723092ffddc93'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:48.807048 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cfc84eb6-cae6-44a5-8360-e3906564c51b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a389b51511644982882723092ffddc93'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:48.807976 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cfc84eb6-cae6-44a5-8360-e3906564c51b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a389b51511644982882723092ffddc93'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:48.824322 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cfc84eb6-cae6-44a5-8360-e3906564c51b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a389b51511644982882723092ffddc93'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:48.838772 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cfc84eb6-cae6-44a5-8360-e3906564c51b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a389b51511644982882723092ffddc93'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:48.856591 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 826/1662] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 03:34:48 2022] DELETE /identity/v3/users/a389b51511644982882723092ffddc93 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:34:48.861148 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f17676-3aab-4097-85f0-6acdf152204e None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:48.861771 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f17676-3aab-4097-85f0-6acdf152204e None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:48.862284 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f17676-3aab-4097-85f0-6acdf152204e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:48.884395 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-19f17676-3aab-4097-85f0-6acdf152204e None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:34:48.886089 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-19f17676-3aab-4097-85f0-6acdf152204e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:48.967709 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 837/1663] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:34:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:34:48.978064 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc84eb6-cae6-44a5-8360-e3906564c51b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:48.979561 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-cfc84eb6-cae6-44a5-8360-e3906564c51b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:49.067535 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc84eb6-cae6-44a5-8360-e3906564c51b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:49.073072 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22edb86a-f067-4f08-ab2d-eb5ab64bfd7a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:49.073623 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22edb86a-f067-4f08-ab2d-eb5ab64bfd7a service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:49.073903 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22edb86a-f067-4f08-ab2d-eb5ab64bfd7a service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:49.074207 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22edb86a-f067-4f08-ab2d-eb5ab64bfd7a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:49.091388 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22edb86a-f067-4f08-ab2d-eb5ab64bfd7a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:49.094170 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22edb86a-f067-4f08-ab2d-eb5ab64bfd7a service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:49.123553 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 827/1664] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:34:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:34:49.595178 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f17676-3aab-4097-85f0-6acdf152204e None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:49.596803 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-19f17676-3aab-4097-85f0-6acdf152204e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:49.610641 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f17676-3aab-4097-85f0-6acdf152204e None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:49.615307 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf03785-b61f-452b-ac29-dade9888e979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:49.615994 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf03785-b61f-452b-ac29-dade9888e979 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:49.616279 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf03785-b61f-452b-ac29-dade9888e979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:49.616553 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf03785-b61f-452b-ac29-dade9888e979 admin admin] PATH_INFO: `/v3/projects/0735003cd30c47298281be972541f8a2` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:49.617665 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf03785-b61f-452b-ac29-dade9888e979 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0735003cd30c47298281be972541f8a2)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:49.619595 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf03785-b61f-452b-ac29-dade9888e979 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:49.672711 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2cf03785-b61f-452b-ac29-dade9888e979 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0735003cd30c47298281be972541f8a2', 'request_id': 'req-2cf03785-b61f-452b-ac29-dade9888e979'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:49.678376 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2cf03785-b61f-452b-ac29-dade9888e979 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0735003cd30c47298281be972541f8a2', 'request_id': 'req-2cf03785-b61f-452b-ac29-dade9888e979'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:49.684809 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2cf03785-b61f-452b-ac29-dade9888e979 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0735003cd30c47298281be972541f8a2', 'request_id': 'req-2cf03785-b61f-452b-ac29-dade9888e979'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:49.697621 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2cf03785-b61f-452b-ac29-dade9888e979 admin admin] The token cache is being invalidate because project 0735003cd30c47298281be972541f8a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 03:34:49.697918 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2cf03785-b61f-452b-ac29-dade9888e979 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:49.698546 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2cf03785-b61f-452b-ac29-dade9888e979 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:49.700073 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 838/1665] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 03:34:49 2022] DELETE /identity/v3/projects/0735003cd30c47298281be972541f8a2 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:34:49.705445 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90736c8d-0dd4-4acb-9a5e-02909bc3ce90 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:49.705907 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90736c8d-0dd4-4acb-9a5e-02909bc3ce90 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:49.706015 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90736c8d-0dd4-4acb-9a5e-02909bc3ce90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:49.725916 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-90736c8d-0dd4-4acb-9a5e-02909bc3ce90 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:34:49.727143 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-90736c8d-0dd4-4acb-9a5e-02909bc3ce90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:49.748653 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:34:49.840217 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 828/1666] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:34:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:34:49.841857 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da4d4c5-634b-4bb4-815d-9d6cae034949 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:49.842141 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da4d4c5-634b-4bb4-815d-9d6cae034949 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:49.842412 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da4d4c5-634b-4bb4-815d-9d6cae034949 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:49.863653 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-3da4d4c5-634b-4bb4-815d-9d6cae034949 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:34:49.864790 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-3da4d4c5-634b-4bb4-815d-9d6cae034949 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:49.874265 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 839/1667] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:34:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:34:49.879079 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90736c8d-0dd4-4acb-9a5e-02909bc3ce90 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:49.899974 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90736c8d-0dd4-4acb-9a5e-02909bc3ce90 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:49.921764 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7cf496-1c4a-4125-905f-82980264c657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:49.923195 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7cf496-1c4a-4125-905f-82980264c657 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:49.928382 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7cf496-1c4a-4125-905f-82980264c657 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:49.928739 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c7cf496-1c4a-4125-905f-82980264c657 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:49.929472 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7cf496-1c4a-4125-905f-82980264c657 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:34:49.929784 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7cf496-1c4a-4125-905f-82980264c657 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:49.942211 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c7cf496-1c4a-4125-905f-82980264c657 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:49.960114 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 829/1668] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 03:34:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:34:49.962416 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da4d4c5-634b-4bb4-815d-9d6cae034949 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:49.972694 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da4d4c5-634b-4bb4-815d-9d6cae034949 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:49.979757 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542d9e71-9465-4845-861e-491f5068737a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:49.980551 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d9e71-9465-4845-861e-491f5068737a admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:49.980908 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d9e71-9465-4845-861e-491f5068737a admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:49.981253 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d9e71-9465-4845-861e-491f5068737a admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:49.981975 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542d9e71-9465-4845-861e-491f5068737a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:49.984036 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542d9e71-9465-4845-861e-491f5068737a admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:50.014889 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 840/1669] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:34:49 2022] POST /identity/v3/projects => generated 381 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:34:50.019204 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7cf496-1c4a-4125-905f-82980264c657 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:50.029845 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c7cf496-1c4a-4125-905f-82980264c657 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:50.103845 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9116f5f-7b6e-49aa-9e1e-ab2640664580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:50.104792 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9116f5f-7b6e-49aa-9e1e-ab2640664580 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:50.105209 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9116f5f-7b6e-49aa-9e1e-ab2640664580 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:50.105582 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9116f5f-7b6e-49aa-9e1e-ab2640664580 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:50.107616 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9116f5f-7b6e-49aa-9e1e-ab2640664580 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:50.110076 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9116f5f-7b6e-49aa-9e1e-ab2640664580 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:50.154339 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 830/1670] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:34:50 2022] POST /identity/v3/users => generated 291 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 03:34:50.155673 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542d9e71-9465-4845-861e-491f5068737a admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:50.171214 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542d9e71-9465-4845-861e-491f5068737a admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:50.178807 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03b472e-e9c9-4b3c-b6bf-33550f39d03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:50.179834 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b472e-e9c9-4b3c-b6bf-33550f39d03c admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:50.180433 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b472e-e9c9-4b3c-b6bf-33550f39d03c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:50.180816 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b472e-e9c9-4b3c-b6bf-33550f39d03c admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:50.181582 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03b472e-e9c9-4b3c-b6bf-33550f39d03c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:50.183621 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03b472e-e9c9-4b3c-b6bf-33550f39d03c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:50.220093 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 841/1671] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:34:50 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:34:50.225744 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9116f5f-7b6e-49aa-9e1e-ab2640664580 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:50.239857 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9116f5f-7b6e-49aa-9e1e-ab2640664580 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:50.249575 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d4d056-0dcc-48c2-8e01-fac6b4d41d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:50.250613 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d4d056-0dcc-48c2-8e01-fac6b4d41d8a admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:50.251050 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d4d056-0dcc-48c2-8e01-fac6b4d41d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:50.251466 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d4d056-0dcc-48c2-8e01-fac6b4d41d8a admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:50.252842 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d4d056-0dcc-48c2-8e01-fac6b4d41d8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:50.254624 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d4d056-0dcc-48c2-8e01-fac6b4d41d8a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:50.274991 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 831/1672] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:34:50 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:34:50.280771 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03b472e-e9c9-4b3c-b6bf-33550f39d03c admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:50.292885 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03b472e-e9c9-4b3c-b6bf-33550f39d03c admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:50.298785 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b84c31-0dc4-49fc-bb4e-2cd9d719b7e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:50.300344 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b84c31-0dc4-49fc-bb4e-2cd9d719b7e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:50.301014 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b84c31-0dc4-49fc-bb4e-2cd9d719b7e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:50.303354 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b84c31-0dc4-49fc-bb4e-2cd9d719b7e8 admin admin] PATH_INFO: `/v3/projects/909fcdad3cef45ad9cf5494524d2b354/users/7f6abf03a1cb4ecaa8b2edfb8a576418/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:50.328247 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b84c31-0dc4-49fc-bb4e-2cd9d719b7e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=909fcdad3cef45ad9cf5494524d2b354, user_id=7f6abf03a1cb4ecaa8b2edfb8a576418, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:50.348178 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b84c31-0dc4-49fc-bb4e-2cd9d719b7e8 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:50.361082 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 842/1673] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:34:50 2022] PUT /identity/v3/projects/909fcdad3cef45ad9cf5494524d2b354/users/7f6abf03a1cb4ecaa8b2edfb8a576418/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:34:50.366567 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd87bc8-cb36-4484-96e5-d4c2661afd67 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:50.366948 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd87bc8-cb36-4484-96e5-d4c2661afd67 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:50.367225 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd87bc8-cb36-4484-96e5-d4c2661afd67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:50.388197 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-ddd87bc8-cb36-4484-96e5-d4c2661afd67 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:34:50.389496 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ddd87bc8-cb36-4484-96e5-d4c2661afd67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:50.473940 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 832/1674] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:34:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:34:50.485433 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b84c31-0dc4-49fc-bb4e-2cd9d719b7e8 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:50.487471 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-86b84c31-0dc4-49fc-bb4e-2cd9d719b7e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:50.515011 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:34:50.571671 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b84c31-0dc4-49fc-bb4e-2cd9d719b7e8 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:50.576652 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe6cb35-7376-4d19-816e-ffaab0f56f0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:50.577327 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe6cb35-7376-4d19-816e-ffaab0f56f0d service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:50.577608 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe6cb35-7376-4d19-816e-ffaab0f56f0d service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:50.577937 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe6cb35-7376-4d19-816e-ffaab0f56f0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:50.589704 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe6cb35-7376-4d19-816e-ffaab0f56f0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:50.591614 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe6cb35-7376-4d19-816e-ffaab0f56f0d service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:50.603644 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 843/1675] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:34:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:34:51.189672 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbf043-d39f-4073-8b13-321a130147e0 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:51.190199 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbf043-d39f-4073-8b13-321a130147e0 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:51.190581 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbf043-d39f-4073-8b13-321a130147e0 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:51.192383 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 833/1676] 10.222.0.26 () {54 vars in 921 bytes} [Wed Nov 2 03:34:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:34:51.201001 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf15d5b5-882d-493c-9b0c-559fd8e602bc None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:51.201597 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf15d5b5-882d-493c-9b0c-559fd8e602bc None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:51.202752 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf15d5b5-882d-493c-9b0c-559fd8e602bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:51.238721 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-cf15d5b5-882d-493c-9b0c-559fd8e602bc None None] MFA Rules not processed for user `9b4b9bd741574ddabadd4b7651ebc885`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:34:51.240515 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-cf15d5b5-882d-493c-9b0c-559fd8e602bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:51.257727 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fbf043-d39f-4073-8b13-321a130147e0 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:51.260323 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-54fbf043-d39f-4073-8b13-321a130147e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:51.297537 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fbf043-d39f-4073-8b13-321a130147e0 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:51.304775 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4516aa07-d170-479e-a916-e4a00c7d0e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:51.305688 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4516aa07-d170-479e-a916-e4a00c7d0e50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:51.306374 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4516aa07-d170-479e-a916-e4a00c7d0e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:51.306684 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4516aa07-d170-479e-a916-e4a00c7d0e50 admin admin] PATH_INFO: `/v3/users/7fdb70174b404f4abf0bc220d5dd4bb1` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:51.308186 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4516aa07-d170-479e-a916-e4a00c7d0e50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fdb70174b404f4abf0bc220d5dd4bb1)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:51.310694 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4516aa07-d170-479e-a916-e4a00c7d0e50 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:51.349962 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:34:51.372255 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 844/1677] 10.222.0.26 () {58 vars in 1018 bytes} [Wed Nov 2 03:34:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:34:51.380051 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7128b-38d5-4c56-ba3d-f25e2085d453 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:51.380697 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7128b-38d5-4c56-ba3d-f25e2085d453 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:51.380995 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad7128b-38d5-4c56-ba3d-f25e2085d453 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:51.384202 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 845/1678] 10.222.0.26 () {54 vars in 961 bytes} [Wed Nov 2 03:34:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:34:51.385480 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4516aa07-d170-479e-a916-e4a00c7d0e50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fdb70174b404f4abf0bc220d5dd4bb1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:51.389907 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee27819-cb86-4008-a819-53305c83637b None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:51.390419 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee27819-cb86-4008-a819-53305c83637b None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:51.390890 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee27819-cb86-4008-a819-53305c83637b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:51.397997 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4516aa07-d170-479e-a916-e4a00c7d0e50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fdb70174b404f4abf0bc220d5dd4bb1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:51.398872 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4516aa07-d170-479e-a916-e4a00c7d0e50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fdb70174b404f4abf0bc220d5dd4bb1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:51.415204 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4516aa07-d170-479e-a916-e4a00c7d0e50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fdb70174b404f4abf0bc220d5dd4bb1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:51.431980 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4516aa07-d170-479e-a916-e4a00c7d0e50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fdb70174b404f4abf0bc220d5dd4bb1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:51.439046 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-7ee27819-cb86-4008-a819-53305c83637b None None] MFA Rules not processed for user `9b4b9bd741574ddabadd4b7651ebc885`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:34:51.440910 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-7ee27819-cb86-4008-a819-53305c83637b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:51.454209 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 834/1679] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 03:34:51 2022] DELETE /identity/v3/users/7fdb70174b404f4abf0bc220d5dd4bb1 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 03:34:51.482228 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e7bdc5-520a-42d1-8304-44bccffcb8c5 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:51.482753 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e7bdc5-520a-42d1-8304-44bccffcb8c5 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:51.483204 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e7bdc5-520a-42d1-8304-44bccffcb8c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:51.526236 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-19e7bdc5-520a-42d1-8304-44bccffcb8c5 None None] MFA Rules not processed for user `7f6abf03a1cb4ecaa8b2edfb8a576418`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:34:51.528064 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-19e7bdc5-520a-42d1-8304-44bccffcb8c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:51.623767 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 846/1680] 10.222.0.26 () {58 vars in 1058 bytes} [Wed Nov 2 03:34:51 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 03:34:51.631733 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee27819-cb86-4008-a819-53305c83637b None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:51.648226 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 835/1681] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:34:51 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:34:51.652117 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee27819-cb86-4008-a819-53305c83637b None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:51.653072 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e7bdc5-520a-42d1-8304-44bccffcb8c5 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:51.655189 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-19e7bdc5-520a-42d1-8304-44bccffcb8c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:51.661895 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088d160e-d878-4a3d-985d-d3c3688d660a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4b9bd741574ddabadd4b7651ebc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:51.662849 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d160e-d878-4a3d-985d-d3c3688d660a service placement] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:51.663355 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d160e-d878-4a3d-985d-d3c3688d660a service placement] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:51.663817 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d160e-d878-4a3d-985d-d3c3688d660a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:51.665750 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-088d160e-d878-4a3d-985d-d3c3688d660a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:51.678636 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088d160e-d878-4a3d-985d-d3c3688d660a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:51.680407 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088d160e-d878-4a3d-985d-d3c3688d660a service placement] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:51.693297 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 847/1682] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:34:51 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:34:51.741420 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088d160e-d878-4a3d-985d-d3c3688d660a service placement] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:51.744005 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e7bdc5-520a-42d1-8304-44bccffcb8c5 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:51.744682 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-088d160e-d878-4a3d-985d-d3c3688d660a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:51.755094 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14bfd69-ad80-4e38-b961-1a27585e67e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:51.755732 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14bfd69-ad80-4e38-b961-1a27585e67e5 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:51.756063 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14bfd69-ad80-4e38-b961-1a27585e67e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:51.756340 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14bfd69-ad80-4e38-b961-1a27585e67e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:51.766642 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14bfd69-ad80-4e38-b961-1a27585e67e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:51.768593 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14bfd69-ad80-4e38-b961-1a27585e67e5 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:51.778715 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 836/1683] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:34:51 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 03:34:51.804062 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:34:51.825546 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088d160e-d878-4a3d-985d-d3c3688d660a service placement] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:51.830685 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c47005d-421b-43ae-9287-a0f47f4836e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:51.831323 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c47005d-421b-43ae-9287-a0f47f4836e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:51.831605 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c47005d-421b-43ae-9287-a0f47f4836e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:51.831878 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c47005d-421b-43ae-9287-a0f47f4836e6 admin admin] PATH_INFO: `/v3/projects/f7ed6d39b2e343548a094be05af24877` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:51.833025 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c47005d-421b-43ae-9287-a0f47f4836e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7ed6d39b2e343548a094be05af24877)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:51.834903 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c47005d-421b-43ae-9287-a0f47f4836e6 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:51.882362 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1c47005d-421b-43ae-9287-a0f47f4836e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7ed6d39b2e343548a094be05af24877', 'request_id': 'req-1c47005d-421b-43ae-9287-a0f47f4836e6'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:51.887900 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1c47005d-421b-43ae-9287-a0f47f4836e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7ed6d39b2e343548a094be05af24877', 'request_id': 'req-1c47005d-421b-43ae-9287-a0f47f4836e6'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:51.894203 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1c47005d-421b-43ae-9287-a0f47f4836e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7ed6d39b2e343548a094be05af24877', 'request_id': 'req-1c47005d-421b-43ae-9287-a0f47f4836e6'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:51.909353 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1c47005d-421b-43ae-9287-a0f47f4836e6 admin admin] The token cache is being invalidate because project f7ed6d39b2e343548a094be05af24877 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 03:34:51.909802 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1c47005d-421b-43ae-9287-a0f47f4836e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:51.910637 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1c47005d-421b-43ae-9287-a0f47f4836e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:34:51.912200 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 848/1684] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 03:34:51 2022] DELETE /identity/v3/projects/f7ed6d39b2e343548a094be05af24877 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 03:34:51.917134 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d01961-b7cb-43e9-b41e-008bd4a28aef None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:51.917422 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d01961-b7cb-43e9-b41e-008bd4a28aef None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:51.917697 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d01961-b7cb-43e9-b41e-008bd4a28aef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:51.941769 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-e6d01961-b7cb-43e9-b41e-008bd4a28aef None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:34:51.943366 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e6d01961-b7cb-43e9-b41e-008bd4a28aef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:51.979493 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:34:52.017531 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 837/1685] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:34:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:34:52.025574 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1142cc-74c1-44f6-a6ae-5d71c604dd67 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:52.025925 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1142cc-74c1-44f6-a6ae-5d71c604dd67 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:52.026982 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1142cc-74c1-44f6-a6ae-5d71c604dd67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:52.054629 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-7d1142cc-74c1-44f6-a6ae-5d71c604dd67 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:34:52.055918 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-7d1142cc-74c1-44f6-a6ae-5d71c604dd67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:52.067992 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 849/1686] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:34:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:34:52.069341 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d01961-b7cb-43e9-b41e-008bd4a28aef None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:52.080185 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d01961-b7cb-43e9-b41e-008bd4a28aef None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:52.101361 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5123541-bcf0-4d9e-827d-96eba80702f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:52.102146 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5123541-bcf0-4d9e-827d-96eba80702f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:52.102798 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5123541-bcf0-4d9e-827d-96eba80702f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:52.103343 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5123541-bcf0-4d9e-827d-96eba80702f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:52.104606 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5123541-bcf0-4d9e-827d-96eba80702f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:34:52.105069 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5123541-bcf0-4d9e-827d-96eba80702f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:52.107924 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5123541-bcf0-4d9e-827d-96eba80702f5 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:52.121640 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 838/1687] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 03:34:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:34:52.136021 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1142cc-74c1-44f6-a6ae-5d71c604dd67 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:52.150374 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1142cc-74c1-44f6-a6ae-5d71c604dd67 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:52.155301 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e323c66-96d3-4b13-bcfc-aff4335baa3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:52.155932 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e323c66-96d3-4b13-bcfc-aff4335baa3e admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:52.156740 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e323c66-96d3-4b13-bcfc-aff4335baa3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:52.157014 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e323c66-96d3-4b13-bcfc-aff4335baa3e admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:52.157709 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e323c66-96d3-4b13-bcfc-aff4335baa3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:52.159600 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e323c66-96d3-4b13-bcfc-aff4335baa3e admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:52.189742 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 850/1688] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:34:52 2022] POST /identity/v3/projects => generated 367 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:34:52.193977 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5123541-bcf0-4d9e-827d-96eba80702f5 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:52.205561 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5123541-bcf0-4d9e-827d-96eba80702f5 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:52.263831 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae2f96b-8323-4d0e-8220-5d5be4921a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:52.264461 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae2f96b-8323-4d0e-8220-5d5be4921a46 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:52.264737 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae2f96b-8323-4d0e-8220-5d5be4921a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:52.265019 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae2f96b-8323-4d0e-8220-5d5be4921a46 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:52.266207 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae2f96b-8323-4d0e-8220-5d5be4921a46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:52.268132 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae2f96b-8323-4d0e-8220-5d5be4921a46 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:52.290562 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 839/1689] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:34:52 2022] POST /identity/v3/users => generated 284 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:34:52.294701 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e323c66-96d3-4b13-bcfc-aff4335baa3e admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:52.305180 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e323c66-96d3-4b13-bcfc-aff4335baa3e admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:52.309839 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07fe961-73a4-412c-baa2-55e17950e9b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:52.310464 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07fe961-73a4-412c-baa2-55e17950e9b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:52.310743 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07fe961-73a4-412c-baa2-55e17950e9b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:52.311013 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07fe961-73a4-412c-baa2-55e17950e9b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:52.311648 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07fe961-73a4-412c-baa2-55e17950e9b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:52.313288 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07fe961-73a4-412c-baa2-55e17950e9b6 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:52.323463 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 851/1690] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:34:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 03:34:52.327304 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae2f96b-8323-4d0e-8220-5d5be4921a46 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:52.338167 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae2f96b-8323-4d0e-8220-5d5be4921a46 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:52.342851 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e7d6b6-7c20-4f71-990a-e4697e3f593e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:52.343470 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e7d6b6-7c20-4f71-990a-e4697e3f593e admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:52.344299 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e7d6b6-7c20-4f71-990a-e4697e3f593e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:52.344594 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e7d6b6-7c20-4f71-990a-e4697e3f593e admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:52.345230 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e7d6b6-7c20-4f71-990a-e4697e3f593e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:52.346817 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e7d6b6-7c20-4f71-990a-e4697e3f593e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:52.358447 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 840/1691] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:34:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:34:52.362622 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07fe961-73a4-412c-baa2-55e17950e9b6 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:52.372536 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07fe961-73a4-412c-baa2-55e17950e9b6 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:52.377392 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422ac7d9-7076-4121-b056-415dc596f6a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:52.378059 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ac7d9-7076-4121-b056-415dc596f6a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:52.378334 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ac7d9-7076-4121-b056-415dc596f6a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:52.378603 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422ac7d9-7076-4121-b056-415dc596f6a7 admin admin] PATH_INFO: `/v3/projects/285a898e09684a11ad62ab603ccf8fa0/users/a48bc4f61b0746c3ba5dd84fb3022081/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:52.393321 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422ac7d9-7076-4121-b056-415dc596f6a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=285a898e09684a11ad62ab603ccf8fa0, user_id=a48bc4f61b0746c3ba5dd84fb3022081, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:52.396076 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422ac7d9-7076-4121-b056-415dc596f6a7 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:52.409942 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 852/1692] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:34:52 2022] PUT /identity/v3/projects/285a898e09684a11ad62ab603ccf8fa0/users/a48bc4f61b0746c3ba5dd84fb3022081/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:34:52.419936 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e07837-f54e-44a2-ae70-7b190dc6ad5c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:52.420252 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e07837-f54e-44a2-ae70-7b190dc6ad5c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:52.420530 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e07837-f54e-44a2-ae70-7b190dc6ad5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:52.448757 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-48e07837-f54e-44a2-ae70-7b190dc6ad5c None None] MFA Rules not processed for user `a48bc4f61b0746c3ba5dd84fb3022081`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:34:52.449954 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-48e07837-f54e-44a2-ae70-7b190dc6ad5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:52.513453 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 841/1693] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:34:52 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:34:52.524426 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422ac7d9-7076-4121-b056-415dc596f6a7 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:34:52.526092 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-422ac7d9-7076-4121-b056-415dc596f6a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:52.582999 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422ac7d9-7076-4121-b056-415dc596f6a7 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:34:52.587116 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5910e5-c09f-46ab-9af1-41b492122832 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:34:52.587827 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5910e5-c09f-46ab-9af1-41b492122832 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:52.588264 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5910e5-c09f-46ab-9af1-41b492122832 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:52.588667 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5910e5-c09f-46ab-9af1-41b492122832 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:52.598033 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5910e5-c09f-46ab-9af1-41b492122832 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:34:52.599513 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5910e5-c09f-46ab-9af1-41b492122832 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:34:52.608323 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 853/1694] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:34:52 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:34:53.195559 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc3ddc-6c24-4b5e-9bd9-c3981423357c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:34:53.196075 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc3ddc-6c24-4b5e-9bd9-c3981423357c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:34:53.196168 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc3ddc-6c24-4b5e-9bd9-c3981423357c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:34:53.215142 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-1afc3ddc-6c24-4b5e-9bd9-c3981423357c None None] MFA Rules not processed for user `7f6abf03a1cb4ecaa8b2edfb8a576418`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:34:53.216367 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1afc3ddc-6c24-4b5e-9bd9-c3981423357c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:34:53.240403 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:34:53.271910 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 842/1695] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:34:53 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:35:01.152190 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5910e5-c09f-46ab-9af1-41b492122832 service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:01.164937 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:35:01.218320 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5910e5-c09f-46ab-9af1-41b492122832 service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:01.222991 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca9346c-4194-4570-8bdf-150c772e67f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:01.223618 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca9346c-4194-4570-8bdf-150c772e67f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:01.223894 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca9346c-4194-4570-8bdf-150c772e67f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:01.224203 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca9346c-4194-4570-8bdf-150c772e67f3 admin admin] PATH_INFO: `/v3/users/a48bc4f61b0746c3ba5dd84fb3022081` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:01.225314 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca9346c-4194-4570-8bdf-150c772e67f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a48bc4f61b0746c3ba5dd84fb3022081)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:01.227220 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca9346c-4194-4570-8bdf-150c772e67f3 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:01.273403 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-cca9346c-4194-4570-8bdf-150c772e67f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a48bc4f61b0746c3ba5dd84fb3022081'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:01.274069 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-cca9346c-4194-4570-8bdf-150c772e67f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a48bc4f61b0746c3ba5dd84fb3022081'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:01.274676 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-cca9346c-4194-4570-8bdf-150c772e67f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a48bc4f61b0746c3ba5dd84fb3022081'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:01.287796 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-cca9346c-4194-4570-8bdf-150c772e67f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a48bc4f61b0746c3ba5dd84fb3022081'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:01.295870 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-cca9346c-4194-4570-8bdf-150c772e67f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a48bc4f61b0746c3ba5dd84fb3022081'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:01.310783 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 854/1696] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 03:35:01 2022] DELETE /identity/v3/users/a48bc4f61b0746c3ba5dd84fb3022081 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:35:01.315515 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17491fb-21e5-49f8-bd49-83853b06173a None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:01.316004 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17491fb-21e5-49f8-bd49-83853b06173a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:01.316136 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17491fb-21e5-49f8-bd49-83853b06173a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:01.334531 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-b17491fb-21e5-49f8-bd49-83853b06173a None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:35:01.335707 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b17491fb-21e5-49f8-bd49-83853b06173a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:01.393349 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 843/1697] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:35:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:35:01.402859 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca9346c-4194-4570-8bdf-150c772e67f3 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:01.404493 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-cca9346c-4194-4570-8bdf-150c772e67f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:01.461007 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca9346c-4194-4570-8bdf-150c772e67f3 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:01.464688 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d842e0-d346-4fa7-9da9-9c2a9ed7382b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:01.465289 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d842e0-d346-4fa7-9da9-9c2a9ed7382b service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:01.465567 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d842e0-d346-4fa7-9da9-9c2a9ed7382b service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:01.465833 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d842e0-d346-4fa7-9da9-9c2a9ed7382b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:01.474837 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d842e0-d346-4fa7-9da9-9c2a9ed7382b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:01.476339 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d842e0-d346-4fa7-9da9-9c2a9ed7382b service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:01.486712 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 855/1698] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:35:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:35:01.777235 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17491fb-21e5-49f8-bd49-83853b06173a None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:01.778961 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b17491fb-21e5-49f8-bd49-83853b06173a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:01.792152 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17491fb-21e5-49f8-bd49-83853b06173a None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:01.797969 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8974d586-0477-46ce-98f8-37d92cc2564c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:01.798613 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8974d586-0477-46ce-98f8-37d92cc2564c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:01.798891 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8974d586-0477-46ce-98f8-37d92cc2564c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:01.799187 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8974d586-0477-46ce-98f8-37d92cc2564c admin admin] PATH_INFO: `/v3/projects/285a898e09684a11ad62ab603ccf8fa0` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:01.800349 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8974d586-0477-46ce-98f8-37d92cc2564c admin admin] RBAC: Authorizing `identity:delete_project(project_id=285a898e09684a11ad62ab603ccf8fa0)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:01.802246 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8974d586-0477-46ce-98f8-37d92cc2564c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:01.855101 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8974d586-0477-46ce-98f8-37d92cc2564c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '285a898e09684a11ad62ab603ccf8fa0', 'request_id': 'req-8974d586-0477-46ce-98f8-37d92cc2564c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:01.861557 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8974d586-0477-46ce-98f8-37d92cc2564c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '285a898e09684a11ad62ab603ccf8fa0', 'request_id': 'req-8974d586-0477-46ce-98f8-37d92cc2564c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:01.869953 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8974d586-0477-46ce-98f8-37d92cc2564c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '285a898e09684a11ad62ab603ccf8fa0', 'request_id': 'req-8974d586-0477-46ce-98f8-37d92cc2564c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:01.883154 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8974d586-0477-46ce-98f8-37d92cc2564c admin admin] The token cache is being invalidate because project 285a898e09684a11ad62ab603ccf8fa0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 03:35:01.883593 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8974d586-0477-46ce-98f8-37d92cc2564c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:01.884831 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8974d586-0477-46ce-98f8-37d92cc2564c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:01.886830 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 844/1699] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 03:35:01 2022] DELETE /identity/v3/projects/285a898e09684a11ad62ab603ccf8fa0 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:35:01.892749 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f20c7-03a6-4f70-920f-21800de2692f None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:01.893043 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f20c7-03a6-4f70-920f-21800de2692f None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:01.893328 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f20c7-03a6-4f70-920f-21800de2692f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:01.915454 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-296f20c7-03a6-4f70-920f-21800de2692f None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:35:01.916751 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-296f20c7-03a6-4f70-920f-21800de2692f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:01.950935 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:35:01.992845 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 856/1700] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:35:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:35:02.001582 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66ef3a-7201-436c-882d-9e3cc71efbc9 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:02.001899 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66ef3a-7201-436c-882d-9e3cc71efbc9 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:02.002180 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d66ef3a-7201-436c-882d-9e3cc71efbc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:02.023393 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-3d66ef3a-7201-436c-882d-9e3cc71efbc9 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:35:02.024687 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3d66ef3a-7201-436c-882d-9e3cc71efbc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:02.032686 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 845/1701] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:35:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 03:35:02.036937 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296f20c7-03a6-4f70-920f-21800de2692f None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:02.047362 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296f20c7-03a6-4f70-920f-21800de2692f None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:02.052583 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e753af5-1266-47a5-8247-8554d11b0d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:02.053197 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e753af5-1266-47a5-8247-8554d11b0d19 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:02.053478 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e753af5-1266-47a5-8247-8554d11b0d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:02.053754 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e753af5-1266-47a5-8247-8554d11b0d19 admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:02.054403 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e753af5-1266-47a5-8247-8554d11b0d19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:35:02.054711 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e753af5-1266-47a5-8247-8554d11b0d19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:02.056335 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e753af5-1266-47a5-8247-8554d11b0d19 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:02.069170 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 857/1702] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 03:35:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:35:02.073493 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d66ef3a-7201-436c-882d-9e3cc71efbc9 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:02.082988 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d66ef3a-7201-436c-882d-9e3cc71efbc9 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:02.088043 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8d4bd2-fba4-45b5-b9b9-69913c37afa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:02.088789 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8d4bd2-fba4-45b5-b9b9-69913c37afa3 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:02.089163 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8d4bd2-fba4-45b5-b9b9-69913c37afa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:02.089497 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8d4bd2-fba4-45b5-b9b9-69913c37afa3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:02.091156 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8d4bd2-fba4-45b5-b9b9-69913c37afa3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:02.093188 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8d4bd2-fba4-45b5-b9b9-69913c37afa3 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:02.120067 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 846/1703] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:35:02 2022] POST /identity/v3/projects => generated 353 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 03:35:02.124432 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e753af5-1266-47a5-8247-8554d11b0d19 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:02.135269 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e753af5-1266-47a5-8247-8554d11b0d19 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:02.191394 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb55726-3397-4dfc-aeca-d47c721bede5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:02.192023 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb55726-3397-4dfc-aeca-d47c721bede5 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:02.192284 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb55726-3397-4dfc-aeca-d47c721bede5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:02.192554 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb55726-3397-4dfc-aeca-d47c721bede5 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:02.193187 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb55726-3397-4dfc-aeca-d47c721bede5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:02.195071 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb55726-3397-4dfc-aeca-d47c721bede5 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:02.214767 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 858/1704] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:35:02 2022] POST /identity/v3/users => generated 277 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:35:02.218826 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8d4bd2-fba4-45b5-b9b9-69913c37afa3 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:02.230105 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8d4bd2-fba4-45b5-b9b9-69913c37afa3 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:02.238761 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef26fc2c-ab2e-409e-bdb4-3ee99c7274bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:02.239308 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef26fc2c-ab2e-409e-bdb4-3ee99c7274bf admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:02.239589 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef26fc2c-ab2e-409e-bdb4-3ee99c7274bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:02.239864 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef26fc2c-ab2e-409e-bdb4-3ee99c7274bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:02.240556 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef26fc2c-ab2e-409e-bdb4-3ee99c7274bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:02.242142 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef26fc2c-ab2e-409e-bdb4-3ee99c7274bf admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:02.253720 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 847/1705] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:35:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:35:02.257300 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb55726-3397-4dfc-aeca-d47c721bede5 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:02.267337 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb55726-3397-4dfc-aeca-d47c721bede5 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:02.272280 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4cd9eb-f79e-4b21-bd58-9dd89e24c4df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:02.272887 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4cd9eb-f79e-4b21-bd58-9dd89e24c4df admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:02.273312 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4cd9eb-f79e-4b21-bd58-9dd89e24c4df admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:02.273611 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4cd9eb-f79e-4b21-bd58-9dd89e24c4df admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:02.274257 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4cd9eb-f79e-4b21-bd58-9dd89e24c4df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:02.275882 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4cd9eb-f79e-4b21-bd58-9dd89e24c4df admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:02.287915 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 859/1706] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:35:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:35:02.291883 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef26fc2c-ab2e-409e-bdb4-3ee99c7274bf admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:02.302741 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef26fc2c-ab2e-409e-bdb4-3ee99c7274bf admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:02.307690 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08a1fe9-01fd-403d-851f-6d58292cf010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:02.308402 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08a1fe9-01fd-403d-851f-6d58292cf010 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:02.308683 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08a1fe9-01fd-403d-851f-6d58292cf010 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:02.308956 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08a1fe9-01fd-403d-851f-6d58292cf010 admin admin] PATH_INFO: `/v3/projects/ee471c51adf245578fd44ac37791dbe9/users/725a3258d5ab4c098e29e3d9a5c5959a/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:02.325032 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08a1fe9-01fd-403d-851f-6d58292cf010 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee471c51adf245578fd44ac37791dbe9, user_id=725a3258d5ab4c098e29e3d9a5c5959a, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:02.327682 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08a1fe9-01fd-403d-851f-6d58292cf010 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:02.329932 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:35:02.344163 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 848/1707] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:35:02 2022] PUT /identity/v3/projects/ee471c51adf245578fd44ac37791dbe9/users/725a3258d5ab4c098e29e3d9a5c5959a/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:35:02.354593 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd517fb-103d-45b8-8e1c-68f48ad5ac9d None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:02.354953 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd517fb-103d-45b8-8e1c-68f48ad5ac9d None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:02.355360 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd517fb-103d-45b8-8e1c-68f48ad5ac9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:02.387251 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-acd517fb-103d-45b8-8e1c-68f48ad5ac9d None None] MFA Rules not processed for user `725a3258d5ab4c098e29e3d9a5c5959a`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:35:02.388503 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-acd517fb-103d-45b8-8e1c-68f48ad5ac9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:02.455840 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 860/1708] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:35:02 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:35:02.460590 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08a1fe9-01fd-403d-851f-6d58292cf010 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:02.522174 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08a1fe9-01fd-403d-851f-6d58292cf010 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:02.527296 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c70721f-195d-43a3-8532-21cf6a09c5f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:02.528015 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70721f-195d-43a3-8532-21cf6a09c5f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:02.528373 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70721f-195d-43a3-8532-21cf6a09c5f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:02.528723 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70721f-195d-43a3-8532-21cf6a09c5f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:02.529435 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c70721f-195d-43a3-8532-21cf6a09c5f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:02.531494 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c70721f-195d-43a3-8532-21cf6a09c5f6 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:02.562496 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 849/1709] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:35:02 2022] POST /identity/v3/projects => generated 355 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:35:02.564164 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd517fb-103d-45b8-8e1c-68f48ad5ac9d None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:02.575676 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd517fb-103d-45b8-8e1c-68f48ad5ac9d None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:02.603286 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:35:02.635253 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0a28e4-a190-4b53-a9d2-cf56233244c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:02.635862 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a28e4-a190-4b53-a9d2-cf56233244c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:02.636235 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a28e4-a190-4b53-a9d2-cf56233244c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:02.636514 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0a28e4-a190-4b53-a9d2-cf56233244c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:02.637149 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0a28e4-a190-4b53-a9d2-cf56233244c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:02.639047 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0a28e4-a190-4b53-a9d2-cf56233244c2 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:02.659885 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 861/1710] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:35:02 2022] POST /identity/v3/users => generated 284 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 03:35:02.663928 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c70721f-195d-43a3-8532-21cf6a09c5f6 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:02.674750 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c70721f-195d-43a3-8532-21cf6a09c5f6 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:02.679420 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58cf06a-e65c-43d5-a648-348a2862ff14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:02.680050 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58cf06a-e65c-43d5-a648-348a2862ff14 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:02.680363 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58cf06a-e65c-43d5-a648-348a2862ff14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:02.680639 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58cf06a-e65c-43d5-a648-348a2862ff14 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:02.681276 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58cf06a-e65c-43d5-a648-348a2862ff14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:02.682853 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58cf06a-e65c-43d5-a648-348a2862ff14 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:02.694737 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 850/1711] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:35:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:35:02.698845 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0a28e4-a190-4b53-a9d2-cf56233244c2 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:02.708757 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0a28e4-a190-4b53-a9d2-cf56233244c2 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:02.713434 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd50ea7-8425-46f8-872d-7eda60b77341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:02.714099 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd50ea7-8425-46f8-872d-7eda60b77341 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:02.714366 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd50ea7-8425-46f8-872d-7eda60b77341 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:02.714642 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd50ea7-8425-46f8-872d-7eda60b77341 admin admin] PATH_INFO: `/v3/projects/de15164e07c24535a93ef2f4b341760e/users/3b088b3ef4df45a8aeadd25953b98f05/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:02.729486 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd50ea7-8425-46f8-872d-7eda60b77341 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de15164e07c24535a93ef2f4b341760e, user_id=3b088b3ef4df45a8aeadd25953b98f05, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:02.732242 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd50ea7-8425-46f8-872d-7eda60b77341 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:02.745361 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 862/1712] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:35:02 2022] PUT /identity/v3/projects/de15164e07c24535a93ef2f4b341760e/users/3b088b3ef4df45a8aeadd25953b98f05/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:35:02.749448 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58cf06a-e65c-43d5-a648-348a2862ff14 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:02.759839 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58cf06a-e65c-43d5-a648-348a2862ff14 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:02.816322 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8a8883-cc87-4c1c-9ba4-cee974ea0f19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:02.816940 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8a8883-cc87-4c1c-9ba4-cee974ea0f19 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:02.817221 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8a8883-cc87-4c1c-9ba4-cee974ea0f19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:02.817495 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8a8883-cc87-4c1c-9ba4-cee974ea0f19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:02.818140 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8a8883-cc87-4c1c-9ba4-cee974ea0f19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:02.819724 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8a8883-cc87-4c1c-9ba4-cee974ea0f19 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:02.829460 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 851/1713] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:35:02 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:35:02.833637 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd50ea7-8425-46f8-872d-7eda60b77341 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:02.843802 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd50ea7-8425-46f8-872d-7eda60b77341 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:02.848370 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9603b55a-4ba6-42e2-90cc-475aa072de22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:02.849010 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9603b55a-4ba6-42e2-90cc-475aa072de22 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:02.849286 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9603b55a-4ba6-42e2-90cc-475aa072de22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:02.849556 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9603b55a-4ba6-42e2-90cc-475aa072de22 admin admin] PATH_INFO: `/v3/projects/de15164e07c24535a93ef2f4b341760e/users/3b088b3ef4df45a8aeadd25953b98f05/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:02.851422 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9603b55a-4ba6-42e2-90cc-475aa072de22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de15164e07c24535a93ef2f4b341760e, user_id=3b088b3ef4df45a8aeadd25953b98f05, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:02.854123 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9603b55a-4ba6-42e2-90cc-475aa072de22 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:02.864528 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 863/1714] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:35:02 2022] PUT /identity/v3/projects/de15164e07c24535a93ef2f4b341760e/users/3b088b3ef4df45a8aeadd25953b98f05/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:35:02.876174 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a47be9-4670-4d10-9025-9ba8222cd180 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:02.876492 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a47be9-4670-4d10-9025-9ba8222cd180 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:02.876771 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a47be9-4670-4d10-9025-9ba8222cd180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:02.904918 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a8a47be9-4670-4d10-9025-9ba8222cd180 None None] MFA Rules not processed for user `3b088b3ef4df45a8aeadd25953b98f05`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:35:02.906682 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a8a47be9-4670-4d10-9025-9ba8222cd180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:02.954827 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:35:02.974838 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 852/1715] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:35:02 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 03:35:02.985874 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9603b55a-4ba6-42e2-90cc-475aa072de22 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:02.987563 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-9603b55a-4ba6-42e2-90cc-475aa072de22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:03.044717 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9603b55a-4ba6-42e2-90cc-475aa072de22 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:03.048875 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7b270b-9764-4e5d-916f-8a69f98392e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:03.049484 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b270b-9764-4e5d-916f-8a69f98392e9 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:03.049763 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b270b-9764-4e5d-916f-8a69f98392e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:03.050034 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b270b-9764-4e5d-916f-8a69f98392e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:03.059068 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7b270b-9764-4e5d-916f-8a69f98392e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:03.060528 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7b270b-9764-4e5d-916f-8a69f98392e9 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:03.070659 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 864/1716] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:35:02 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:35:03.145225 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a47be9-4670-4d10-9025-9ba8222cd180 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:03.155400 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a47be9-4670-4d10-9025-9ba8222cd180 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:03.159717 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d6293c-0ef8-4c97-b110-c00f274c7de4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:03.160362 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d6293c-0ef8-4c97-b110-c00f274c7de4 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:03.160640 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d6293c-0ef8-4c97-b110-c00f274c7de4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:03.160915 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d6293c-0ef8-4c97-b110-c00f274c7de4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:03.219448 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d6293c-0ef8-4c97-b110-c00f274c7de4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:03.220913 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d6293c-0ef8-4c97-b110-c00f274c7de4 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:03.230005 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 853/1717] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:35:03 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:35:11.367114 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7b270b-9764-4e5d-916f-8a69f98392e9 service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:11.369205 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-2c7b270b-9764-4e5d-916f-8a69f98392e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:11.435189 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7b270b-9764-4e5d-916f-8a69f98392e9 service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:11.440449 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be56f3f-75b4-4260-8a3d-cbb014155bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:11.441109 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be56f3f-75b4-4260-8a3d-cbb014155bfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:11.441387 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be56f3f-75b4-4260-8a3d-cbb014155bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:11.441666 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be56f3f-75b4-4260-8a3d-cbb014155bfa admin admin] PATH_INFO: `/v3/users/149624e2c0584cf39b6cc468668bc4d1` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:11.442787 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be56f3f-75b4-4260-8a3d-cbb014155bfa admin admin] RBAC: Authorizing `identity:delete_user(user_id=149624e2c0584cf39b6cc468668bc4d1)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:11.444875 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be56f3f-75b4-4260-8a3d-cbb014155bfa admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:11.475795 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:35:11.498240 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5be56f3f-75b4-4260-8a3d-cbb014155bfa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '149624e2c0584cf39b6cc468668bc4d1'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:11.499200 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5be56f3f-75b4-4260-8a3d-cbb014155bfa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '149624e2c0584cf39b6cc468668bc4d1'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:11.500479 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5be56f3f-75b4-4260-8a3d-cbb014155bfa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '149624e2c0584cf39b6cc468668bc4d1'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:11.514388 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5be56f3f-75b4-4260-8a3d-cbb014155bfa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '149624e2c0584cf39b6cc468668bc4d1'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:11.524932 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5be56f3f-75b4-4260-8a3d-cbb014155bfa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '149624e2c0584cf39b6cc468668bc4d1'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:11.543000 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 865/1718] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 03:35:11 2022] DELETE /identity/v3/users/149624e2c0584cf39b6cc468668bc4d1 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:35:11.547684 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a5d88-e66f-4636-be78-72efff5b0cb1 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:11.548205 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a5d88-e66f-4636-be78-72efff5b0cb1 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:11.548702 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72a5d88-e66f-4636-be78-72efff5b0cb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:11.568806 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-f72a5d88-e66f-4636-be78-72efff5b0cb1 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:35:11.570010 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f72a5d88-e66f-4636-be78-72efff5b0cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:11.658813 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 854/1719] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:35:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:35:11.675032 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be56f3f-75b4-4260-8a3d-cbb014155bfa admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:11.678420 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-5be56f3f-75b4-4260-8a3d-cbb014155bfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:11.750876 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be56f3f-75b4-4260-8a3d-cbb014155bfa admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:11.754866 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4642a279-08db-403c-8b37-152a948808a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:11.755527 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4642a279-08db-403c-8b37-152a948808a1 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:11.755837 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4642a279-08db-403c-8b37-152a948808a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:11.756143 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4642a279-08db-403c-8b37-152a948808a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:11.765765 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4642a279-08db-403c-8b37-152a948808a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:11.767230 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4642a279-08db-403c-8b37-152a948808a1 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:11.778073 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 866/1720] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:35:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:35:12.045115 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72a5d88-e66f-4636-be78-72efff5b0cb1 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:12.047097 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f72a5d88-e66f-4636-be78-72efff5b0cb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:12.060579 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72a5d88-e66f-4636-be78-72efff5b0cb1 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:12.065742 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba5d3e7-eda8-4334-a41f-842be19fa47e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:12.066381 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba5d3e7-eda8-4334-a41f-842be19fa47e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:12.066655 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba5d3e7-eda8-4334-a41f-842be19fa47e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:12.066926 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba5d3e7-eda8-4334-a41f-842be19fa47e admin admin] PATH_INFO: `/v3/projects/6c1510f29e8a4608992064d6d60f2951` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:12.068840 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba5d3e7-eda8-4334-a41f-842be19fa47e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c1510f29e8a4608992064d6d60f2951)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:12.071121 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba5d3e7-eda8-4334-a41f-842be19fa47e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:12.118968 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1ba5d3e7-eda8-4334-a41f-842be19fa47e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c1510f29e8a4608992064d6d60f2951', 'request_id': 'req-1ba5d3e7-eda8-4334-a41f-842be19fa47e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:12.125763 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1ba5d3e7-eda8-4334-a41f-842be19fa47e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c1510f29e8a4608992064d6d60f2951', 'request_id': 'req-1ba5d3e7-eda8-4334-a41f-842be19fa47e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:12.133158 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1ba5d3e7-eda8-4334-a41f-842be19fa47e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c1510f29e8a4608992064d6d60f2951', 'request_id': 'req-1ba5d3e7-eda8-4334-a41f-842be19fa47e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:12.144654 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1ba5d3e7-eda8-4334-a41f-842be19fa47e admin admin] The token cache is being invalidate because project 6c1510f29e8a4608992064d6d60f2951 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 03:35:12.145004 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1ba5d3e7-eda8-4334-a41f-842be19fa47e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:12.146291 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1ba5d3e7-eda8-4334-a41f-842be19fa47e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:12.148699 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 855/1721] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 03:35:12 2022] DELETE /identity/v3/projects/6c1510f29e8a4608992064d6d60f2951 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:35:12.154941 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d043992-72fe-480c-88c1-8bc350f16de5 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:12.155240 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d043992-72fe-480c-88c1-8bc350f16de5 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:12.156045 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d043992-72fe-480c-88c1-8bc350f16de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:12.178077 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-0d043992-72fe-480c-88c1-8bc350f16de5 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:35:12.179259 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-0d043992-72fe-480c-88c1-8bc350f16de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:12.239472 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 867/1722] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:35:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:35:12.247774 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec896e-5e55-46fd-83d6-41e21c4d4126 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:12.248100 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec896e-5e55-46fd-83d6-41e21c4d4126 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:12.248629 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec896e-5e55-46fd-83d6-41e21c4d4126 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:12.267967 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-03ec896e-5e55-46fd-83d6-41e21c4d4126 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:35:12.269178 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-03ec896e-5e55-46fd-83d6-41e21c4d4126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:12.277444 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 856/1723] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:35:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:35:12.282797 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d043992-72fe-480c-88c1-8bc350f16de5 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:12.294331 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d043992-72fe-480c-88c1-8bc350f16de5 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:12.299780 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217a4b39-6ab0-4913-b9d1-c04d21639cab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:12.300442 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a4b39-6ab0-4913-b9d1-c04d21639cab admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:12.300718 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a4b39-6ab0-4913-b9d1-c04d21639cab admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:12.300990 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-217a4b39-6ab0-4913-b9d1-c04d21639cab admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:12.301639 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217a4b39-6ab0-4913-b9d1-c04d21639cab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:35:12.301963 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217a4b39-6ab0-4913-b9d1-c04d21639cab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:12.303547 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-217a4b39-6ab0-4913-b9d1-c04d21639cab admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:12.315258 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 868/1724] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 03:35:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:35:12.319775 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ec896e-5e55-46fd-83d6-41e21c4d4126 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:12.325421 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:35:12.329359 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ec896e-5e55-46fd-83d6-41e21c4d4126 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:12.335652 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24909e09-86cf-4a25-9c17-2a826a557ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:12.336297 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24909e09-86cf-4a25-9c17-2a826a557ead admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:12.336766 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24909e09-86cf-4a25-9c17-2a826a557ead admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:12.337210 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24909e09-86cf-4a25-9c17-2a826a557ead admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:12.338103 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24909e09-86cf-4a25-9c17-2a826a557ead admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:12.340052 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24909e09-86cf-4a25-9c17-2a826a557ead admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:12.367590 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 857/1725] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:35:12 2022] POST /identity/v3/projects => generated 359 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 03:35:12.372737 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217a4b39-6ab0-4913-b9d1-c04d21639cab admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:12.384657 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-217a4b39-6ab0-4913-b9d1-c04d21639cab admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:12.417377 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:35:12.447776 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3626fb-d81c-4347-a3e9-40fe3de9348b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:12.448432 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3626fb-d81c-4347-a3e9-40fe3de9348b admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:12.448716 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3626fb-d81c-4347-a3e9-40fe3de9348b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:12.449023 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3626fb-d81c-4347-a3e9-40fe3de9348b admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:12.449661 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3626fb-d81c-4347-a3e9-40fe3de9348b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:12.451582 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3626fb-d81c-4347-a3e9-40fe3de9348b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:12.473673 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 869/1726] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:35:12 2022] POST /identity/v3/users => generated 280 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:35:12.478099 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24909e09-86cf-4a25-9c17-2a826a557ead admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:12.489319 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24909e09-86cf-4a25-9c17-2a826a557ead admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:12.494325 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6c8db5-a642-4731-a659-7c46e63ab883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:12.494957 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6c8db5-a642-4731-a659-7c46e63ab883 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:12.495237 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6c8db5-a642-4731-a659-7c46e63ab883 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:12.495510 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6c8db5-a642-4731-a659-7c46e63ab883 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:12.496203 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6c8db5-a642-4731-a659-7c46e63ab883 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:12.498128 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6c8db5-a642-4731-a659-7c46e63ab883 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:12.509067 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 858/1727] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:35:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:35:12.513246 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3626fb-d81c-4347-a3e9-40fe3de9348b admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:12.523966 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3626fb-d81c-4347-a3e9-40fe3de9348b admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:12.529899 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b93b5e0-ac2c-4f05-8fab-d8de6329d39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:12.530504 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b93b5e0-ac2c-4f05-8fab-d8de6329d39e admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:12.530779 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b93b5e0-ac2c-4f05-8fab-d8de6329d39e admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:12.531057 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b93b5e0-ac2c-4f05-8fab-d8de6329d39e admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:12.531694 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b93b5e0-ac2c-4f05-8fab-d8de6329d39e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:12.533370 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b93b5e0-ac2c-4f05-8fab-d8de6329d39e admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:12.546919 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 870/1728] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:35:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:35:12.551248 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6c8db5-a642-4731-a659-7c46e63ab883 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:12.561518 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6c8db5-a642-4731-a659-7c46e63ab883 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:12.566756 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca804ba-b009-40bf-b503-c34ae129cb80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:12.567406 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca804ba-b009-40bf-b503-c34ae129cb80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:12.567687 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca804ba-b009-40bf-b503-c34ae129cb80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:12.567987 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca804ba-b009-40bf-b503-c34ae129cb80 admin admin] PATH_INFO: `/v3/projects/6c13c7bd1b644e44a33048e08ab88124/users/199574d41db74b858098025fe055c55e/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:12.582541 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca804ba-b009-40bf-b503-c34ae129cb80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c13c7bd1b644e44a33048e08ab88124, user_id=199574d41db74b858098025fe055c55e, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:12.585292 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca804ba-b009-40bf-b503-c34ae129cb80 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:12.598245 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 859/1729] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:35:12 2022] PUT /identity/v3/projects/6c13c7bd1b644e44a33048e08ab88124/users/199574d41db74b858098025fe055c55e/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:35:12.610303 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a86cac-b3ae-484d-b4b6-9c53f72574cd None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:12.610593 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a86cac-b3ae-484d-b4b6-9c53f72574cd None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:12.611027 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a86cac-b3ae-484d-b4b6-9c53f72574cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:12.641299 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-c4a86cac-b3ae-484d-b4b6-9c53f72574cd None None] MFA Rules not processed for user `199574d41db74b858098025fe055c55e`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:35:12.642508 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-c4a86cac-b3ae-484d-b4b6-9c53f72574cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:12.708306 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 871/1730] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:35:12 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:35:12.744348 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca804ba-b009-40bf-b503-c34ae129cb80 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:12.746101 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3ca804ba-b009-40bf-b503-c34ae129cb80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:12.808294 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca804ba-b009-40bf-b503-c34ae129cb80 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:12.812676 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a189da2b-de30-40bc-83c9-5c4c851cee44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:12.813299 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a189da2b-de30-40bc-83c9-5c4c851cee44 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:12.813579 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a189da2b-de30-40bc-83c9-5c4c851cee44 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:12.813852 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a189da2b-de30-40bc-83c9-5c4c851cee44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:12.823022 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a189da2b-de30-40bc-83c9-5c4c851cee44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:12.824524 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a189da2b-de30-40bc-83c9-5c4c851cee44 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:12.833834 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 860/1731] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:35:12 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:35:12.850134 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a86cac-b3ae-484d-b4b6-9c53f72574cd None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:12.935740 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a86cac-b3ae-484d-b4b6-9c53f72574cd None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:12.943672 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d43c4de-f280-4ada-8f89-8d042595acf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:12.944359 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d43c4de-f280-4ada-8f89-8d042595acf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:12.944727 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d43c4de-f280-4ada-8f89-8d042595acf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:12.945500 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d43c4de-f280-4ada-8f89-8d042595acf7 admin admin] PATH_INFO: `/v3/users/199574d41db74b858098025fe055c55e` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:12.946736 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d43c4de-f280-4ada-8f89-8d042595acf7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=199574d41db74b858098025fe055c55e)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:12.948298 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d43c4de-f280-4ada-8f89-8d042595acf7 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:13.010480 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:35:13.015089 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3d43c4de-f280-4ada-8f89-8d042595acf7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '199574d41db74b858098025fe055c55e'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:13.016895 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3d43c4de-f280-4ada-8f89-8d042595acf7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '199574d41db74b858098025fe055c55e'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:13.017602 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3d43c4de-f280-4ada-8f89-8d042595acf7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '199574d41db74b858098025fe055c55e'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:13.039823 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3d43c4de-f280-4ada-8f89-8d042595acf7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '199574d41db74b858098025fe055c55e'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:13.061164 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3d43c4de-f280-4ada-8f89-8d042595acf7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '199574d41db74b858098025fe055c55e'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:13.076269 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 872/1732] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 03:35:12 2022] DELETE /identity/v3/users/199574d41db74b858098025fe055c55e => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:35:13.080836 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe802f-74b6-4ebc-845c-4c2d8f3ece09 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:13.081124 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe802f-74b6-4ebc-845c-4c2d8f3ece09 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:13.081410 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefe802f-74b6-4ebc-845c-4c2d8f3ece09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:13.116119 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-aefe802f-74b6-4ebc-845c-4c2d8f3ece09 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:35:13.117354 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-aefe802f-74b6-4ebc-845c-4c2d8f3ece09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:13.184446 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 861/1733] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:35:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:35:13.194076 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d43c4de-f280-4ada-8f89-8d042595acf7 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:13.195510 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-3d43c4de-f280-4ada-8f89-8d042595acf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:13.254098 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d43c4de-f280-4ada-8f89-8d042595acf7 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:13.257717 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bb0420-08cc-488d-bde1-fd13adf17c52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:13.258323 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bb0420-08cc-488d-bde1-fd13adf17c52 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:13.258602 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bb0420-08cc-488d-bde1-fd13adf17c52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:13.258881 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bb0420-08cc-488d-bde1-fd13adf17c52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:13.268411 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bb0420-08cc-488d-bde1-fd13adf17c52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:13.269877 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bb0420-08cc-488d-bde1-fd13adf17c52 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:13.279494 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 873/1734] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:35:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:35:13.549444 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe802f-74b6-4ebc-845c-4c2d8f3ece09 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:13.550866 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-aefe802f-74b6-4ebc-845c-4c2d8f3ece09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:13.564936 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefe802f-74b6-4ebc-845c-4c2d8f3ece09 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:13.569574 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8183ea-165b-4c4c-b7dd-309e93a524c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:13.570730 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8183ea-165b-4c4c-b7dd-309e93a524c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:13.571191 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8183ea-165b-4c4c-b7dd-309e93a524c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:13.571473 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8183ea-165b-4c4c-b7dd-309e93a524c8 admin admin] PATH_INFO: `/v3/projects/6c13c7bd1b644e44a33048e08ab88124` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:13.572638 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8183ea-165b-4c4c-b7dd-309e93a524c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c13c7bd1b644e44a33048e08ab88124)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:13.574527 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8183ea-165b-4c4c-b7dd-309e93a524c8 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:13.620518 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:35:13.622883 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ad8183ea-165b-4c4c-b7dd-309e93a524c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c13c7bd1b644e44a33048e08ab88124', 'request_id': 'req-ad8183ea-165b-4c4c-b7dd-309e93a524c8'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:13.631585 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ad8183ea-165b-4c4c-b7dd-309e93a524c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c13c7bd1b644e44a33048e08ab88124', 'request_id': 'req-ad8183ea-165b-4c4c-b7dd-309e93a524c8'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:13.640234 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ad8183ea-165b-4c4c-b7dd-309e93a524c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c13c7bd1b644e44a33048e08ab88124', 'request_id': 'req-ad8183ea-165b-4c4c-b7dd-309e93a524c8'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:13.650406 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ad8183ea-165b-4c4c-b7dd-309e93a524c8 admin admin] The token cache is being invalidate because project 6c13c7bd1b644e44a33048e08ab88124 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 03:35:13.650694 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ad8183ea-165b-4c4c-b7dd-309e93a524c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:13.651320 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ad8183ea-165b-4c4c-b7dd-309e93a524c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:13.652877 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 862/1735] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 03:35:13 2022] DELETE /identity/v3/projects/6c13c7bd1b644e44a33048e08ab88124 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 03:35:13.658478 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ed2fbf-4a55-48a6-80fe-f49b55838fd8 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:13.658945 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ed2fbf-4a55-48a6-80fe-f49b55838fd8 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:13.659037 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ed2fbf-4a55-48a6-80fe-f49b55838fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:13.678986 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-29ed2fbf-4a55-48a6-80fe-f49b55838fd8 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:35:13.680215 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-29ed2fbf-4a55-48a6-80fe-f49b55838fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:13.739084 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 874/1736] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:35:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:35:13.749694 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37d5c4-64d9-4c03-b7e4-dbd26f9bb269 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:13.749977 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37d5c4-64d9-4c03-b7e4-dbd26f9bb269 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:13.750251 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e37d5c4-64d9-4c03-b7e4-dbd26f9bb269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:13.769781 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-6e37d5c4-64d9-4c03-b7e4-dbd26f9bb269 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:35:13.770997 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6e37d5c4-64d9-4c03-b7e4-dbd26f9bb269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:13.778868 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 863/1737] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:35:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:35:13.783678 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ed2fbf-4a55-48a6-80fe-f49b55838fd8 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:13.794550 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ed2fbf-4a55-48a6-80fe-f49b55838fd8 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:13.799854 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e9482-9ce5-4dda-bb9e-523afc42a7bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:13.800520 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e9482-9ce5-4dda-bb9e-523afc42a7bb admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:13.800800 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e9482-9ce5-4dda-bb9e-523afc42a7bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:13.801075 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752e9482-9ce5-4dda-bb9e-523afc42a7bb admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:13.801823 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752e9482-9ce5-4dda-bb9e-523afc42a7bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:35:13.802058 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752e9482-9ce5-4dda-bb9e-523afc42a7bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:13.803659 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752e9482-9ce5-4dda-bb9e-523afc42a7bb admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:13.819406 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 875/1738] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 03:35:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:35:13.820892 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e37d5c4-64d9-4c03-b7e4-dbd26f9bb269 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:13.830511 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e37d5c4-64d9-4c03-b7e4-dbd26f9bb269 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:13.835167 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4402e8-d78e-45b7-b8f7-122f36bea8b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:13.835857 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4402e8-d78e-45b7-b8f7-122f36bea8b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:13.836184 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4402e8-d78e-45b7-b8f7-122f36bea8b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:13.836460 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4402e8-d78e-45b7-b8f7-122f36bea8b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:13.837092 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4402e8-d78e-45b7-b8f7-122f36bea8b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:13.838984 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4402e8-d78e-45b7-b8f7-122f36bea8b1 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:13.866878 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 864/1739] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:35:13 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:35:13.871292 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e9482-9ce5-4dda-bb9e-523afc42a7bb admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:13.881692 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752e9482-9ce5-4dda-bb9e-523afc42a7bb admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:13.928456 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:35:13.940041 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686e3199-0fa7-438e-9343-ba83183eb566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:13.940651 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e3199-0fa7-438e-9343-ba83183eb566 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:13.940928 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e3199-0fa7-438e-9343-ba83183eb566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:13.941199 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686e3199-0fa7-438e-9343-ba83183eb566 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:13.941831 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686e3199-0fa7-438e-9343-ba83183eb566 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:13.943759 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-686e3199-0fa7-438e-9343-ba83183eb566 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:13.965195 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 876/1740] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:35:13 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:35:13.969770 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4402e8-d78e-45b7-b8f7-122f36bea8b1 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:13.979909 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4402e8-d78e-45b7-b8f7-122f36bea8b1 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:13.985115 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a311642-3cf7-49c3-9ab9-17326f221322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:13.985717 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a311642-3cf7-49c3-9ab9-17326f221322 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:13.985992 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a311642-3cf7-49c3-9ab9-17326f221322 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:13.986364 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a311642-3cf7-49c3-9ab9-17326f221322 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:13.986997 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a311642-3cf7-49c3-9ab9-17326f221322 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:13.988904 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a311642-3cf7-49c3-9ab9-17326f221322 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:14.000098 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 865/1741] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:35:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 03:35:14.004240 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686e3199-0fa7-438e-9343-ba83183eb566 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:14.014482 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686e3199-0fa7-438e-9343-ba83183eb566 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:14.019597 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8504e4-80d8-4d3b-b051-705db934d1a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:14.020227 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8504e4-80d8-4d3b-b051-705db934d1a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:14.020514 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8504e4-80d8-4d3b-b051-705db934d1a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:14.020791 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8504e4-80d8-4d3b-b051-705db934d1a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:14.021429 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8504e4-80d8-4d3b-b051-705db934d1a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:14.023030 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8504e4-80d8-4d3b-b051-705db934d1a7 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:14.036091 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 877/1742] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:35:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:35:14.040241 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a311642-3cf7-49c3-9ab9-17326f221322 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:14.051184 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a311642-3cf7-49c3-9ab9-17326f221322 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:14.056791 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fb7023-cc7a-4c56-a114-14918b3138cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:14.057470 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fb7023-cc7a-4c56-a114-14918b3138cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:14.057748 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fb7023-cc7a-4c56-a114-14918b3138cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:14.058028 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fb7023-cc7a-4c56-a114-14918b3138cd admin admin] PATH_INFO: `/v3/projects/7f8277f6dfaf4d6bb5e2ece4d9b54b3b/users/f48848b5523c41c1a441a3a87e6fc149/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:14.075607 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fb7023-cc7a-4c56-a114-14918b3138cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f8277f6dfaf4d6bb5e2ece4d9b54b3b, user_id=f48848b5523c41c1a441a3a87e6fc149, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:14.078299 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1fb7023-cc7a-4c56-a114-14918b3138cd admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:14.093085 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 866/1743] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:35:14 2022] PUT /identity/v3/projects/7f8277f6dfaf4d6bb5e2ece4d9b54b3b/users/f48848b5523c41c1a441a3a87e6fc149/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 03:35:14.103444 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a20e97-6e4d-4c74-a2af-3cba51df7606 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:14.104033 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a20e97-6e4d-4c74-a2af-3cba51df7606 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:14.104460 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a20e97-6e4d-4c74-a2af-3cba51df7606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:14.142689 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-e8a20e97-6e4d-4c74-a2af-3cba51df7606 None None] MFA Rules not processed for user `f48848b5523c41c1a441a3a87e6fc149`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:35:14.143912 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e8a20e97-6e4d-4c74-a2af-3cba51df7606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:14.226870 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 878/1744] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:35:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 03:35:14.231437 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fb7023-cc7a-4c56-a114-14918b3138cd admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:14.306390 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1fb7023-cc7a-4c56-a114-14918b3138cd admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:14.312411 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6745c20c-4b0e-4230-867e-22734834ead9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:14.313035 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6745c20c-4b0e-4230-867e-22734834ead9 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:14.313310 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6745c20c-4b0e-4230-867e-22734834ead9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:14.313581 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6745c20c-4b0e-4230-867e-22734834ead9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:14.314600 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6745c20c-4b0e-4230-867e-22734834ead9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:14.316332 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6745c20c-4b0e-4230-867e-22734834ead9 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:14.353155 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 867/1745] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:35:14 2022] POST /identity/v3/projects => generated 375 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 03:35:14.356800 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a20e97-6e4d-4c74-a2af-3cba51df7606 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:14.369089 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a20e97-6e4d-4c74-a2af-3cba51df7606 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:14.446638 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d313330-3c2a-4937-b4eb-a6701f59ee49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:14.447248 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d313330-3c2a-4937-b4eb-a6701f59ee49 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:14.447526 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d313330-3c2a-4937-b4eb-a6701f59ee49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:14.447797 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d313330-3c2a-4937-b4eb-a6701f59ee49 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:14.448457 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d313330-3c2a-4937-b4eb-a6701f59ee49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:14.450348 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d313330-3c2a-4937-b4eb-a6701f59ee49 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:14.470854 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 879/1746] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:35:14 2022] POST /identity/v3/users => generated 294 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:35:14.475322 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6745c20c-4b0e-4230-867e-22734834ead9 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:14.486217 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6745c20c-4b0e-4230-867e-22734834ead9 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:14.490852 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0b0142-bf20-4487-8553-a98b8a7320aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:14.491458 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0b0142-bf20-4487-8553-a98b8a7320aa admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:14.491739 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0b0142-bf20-4487-8553-a98b8a7320aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:14.492055 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0b0142-bf20-4487-8553-a98b8a7320aa admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:14.492807 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0b0142-bf20-4487-8553-a98b8a7320aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:14.494402 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0b0142-bf20-4487-8553-a98b8a7320aa admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:14.505889 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 868/1747] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:35:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:35:14.510083 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d313330-3c2a-4937-b4eb-a6701f59ee49 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:14.520072 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d313330-3c2a-4937-b4eb-a6701f59ee49 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:14.525002 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db19d44-d306-4f59-bdfe-ecee94508f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:14.525670 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db19d44-d306-4f59-bdfe-ecee94508f45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:14.525945 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db19d44-d306-4f59-bdfe-ecee94508f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:14.526223 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db19d44-d306-4f59-bdfe-ecee94508f45 admin admin] PATH_INFO: `/v3/projects/c15b76e29c2e4bec99b8e00d6ee0ec57/users/17d1195cbcb4455884810966956b2dd0/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:14.543258 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db19d44-d306-4f59-bdfe-ecee94508f45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c15b76e29c2e4bec99b8e00d6ee0ec57, user_id=17d1195cbcb4455884810966956b2dd0, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:14.545977 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db19d44-d306-4f59-bdfe-ecee94508f45 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:14.559266 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 880/1748] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:35:14 2022] PUT /identity/v3/projects/c15b76e29c2e4bec99b8e00d6ee0ec57/users/17d1195cbcb4455884810966956b2dd0/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:35:14.564057 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0b0142-bf20-4487-8553-a98b8a7320aa admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:14.574181 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0b0142-bf20-4487-8553-a98b8a7320aa admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:14.616312 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:35:14.635557 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0b9cf6-e3d1-4f9b-9932-bd3b4ed8855f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:14.636220 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b9cf6-e3d1-4f9b-9932-bd3b4ed8855f admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:14.636498 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b9cf6-e3d1-4f9b-9932-bd3b4ed8855f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:14.636769 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b9cf6-e3d1-4f9b-9932-bd3b4ed8855f admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:14.637412 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0b9cf6-e3d1-4f9b-9932-bd3b4ed8855f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:14.639008 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0b9cf6-e3d1-4f9b-9932-bd3b4ed8855f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:14.649880 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 869/1749] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:35:14 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:35:14.654290 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db19d44-d306-4f59-bdfe-ecee94508f45 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:14.664912 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db19d44-d306-4f59-bdfe-ecee94508f45 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:14.669765 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bd0a5c-bb7c-4113-a0d4-b5f191fb4e12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:14.670421 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd0a5c-bb7c-4113-a0d4-b5f191fb4e12 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:14.670700 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd0a5c-bb7c-4113-a0d4-b5f191fb4e12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:14.670971 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bd0a5c-bb7c-4113-a0d4-b5f191fb4e12 admin admin] PATH_INFO: `/v3/projects/c15b76e29c2e4bec99b8e00d6ee0ec57/users/17d1195cbcb4455884810966956b2dd0/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:14.673464 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bd0a5c-bb7c-4113-a0d4-b5f191fb4e12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c15b76e29c2e4bec99b8e00d6ee0ec57, user_id=17d1195cbcb4455884810966956b2dd0, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:14.676162 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bd0a5c-bb7c-4113-a0d4-b5f191fb4e12 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:14.688008 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 881/1750] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:35:14 2022] PUT /identity/v3/projects/c15b76e29c2e4bec99b8e00d6ee0ec57/users/17d1195cbcb4455884810966956b2dd0/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:35:14.698961 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d3b694-96fb-48d4-83cb-0ffaa6655050 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:14.699249 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d3b694-96fb-48d4-83cb-0ffaa6655050 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:14.699524 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d3b694-96fb-48d4-83cb-0ffaa6655050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:14.727659 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-f7d3b694-96fb-48d4-83cb-0ffaa6655050 None None] MFA Rules not processed for user `17d1195cbcb4455884810966956b2dd0`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:35:14.729043 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f7d3b694-96fb-48d4-83cb-0ffaa6655050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:14.802744 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 870/1751] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:35:14 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:35:14.813538 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bd0a5c-bb7c-4113-a0d4-b5f191fb4e12 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:14.815260 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a8bd0a5c-bb7c-4113-a0d4-b5f191fb4e12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:14.829195 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:35:14.875287 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bd0a5c-bb7c-4113-a0d4-b5f191fb4e12 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:14.878894 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2825fe1b-2ced-4855-88a0-20bca5000046 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:14.879493 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2825fe1b-2ced-4855-88a0-20bca5000046 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:14.879766 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2825fe1b-2ced-4855-88a0-20bca5000046 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:14.880071 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2825fe1b-2ced-4855-88a0-20bca5000046 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:14.936765 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2825fe1b-2ced-4855-88a0-20bca5000046 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:14.938130 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2825fe1b-2ced-4855-88a0-20bca5000046 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:14.946911 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 882/1752] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:35:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:35:17.508432 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d3b694-96fb-48d4-83cb-0ffaa6655050 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:17.571286 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d3b694-96fb-48d4-83cb-0ffaa6655050 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:17.575831 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbfb5b9-ad26-43ca-8b4f-6c4474af94d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:17.576491 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbfb5b9-ad26-43ca-8b4f-6c4474af94d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:17.576771 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbfb5b9-ad26-43ca-8b4f-6c4474af94d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:17.577049 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbfb5b9-ad26-43ca-8b4f-6c4474af94d6 admin admin] PATH_INFO: `/v3/users/f48848b5523c41c1a441a3a87e6fc149` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:17.578188 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbfb5b9-ad26-43ca-8b4f-6c4474af94d6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f48848b5523c41c1a441a3a87e6fc149)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:17.580110 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbfb5b9-ad26-43ca-8b4f-6c4474af94d6 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:17.630354 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-dcbfb5b9-ad26-43ca-8b4f-6c4474af94d6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f48848b5523c41c1a441a3a87e6fc149'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:17.637384 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-dcbfb5b9-ad26-43ca-8b4f-6c4474af94d6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f48848b5523c41c1a441a3a87e6fc149'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:17.638044 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-dcbfb5b9-ad26-43ca-8b4f-6c4474af94d6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f48848b5523c41c1a441a3a87e6fc149'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:17.646279 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:35:17.651216 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-dcbfb5b9-ad26-43ca-8b4f-6c4474af94d6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f48848b5523c41c1a441a3a87e6fc149'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:17.658722 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-dcbfb5b9-ad26-43ca-8b4f-6c4474af94d6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f48848b5523c41c1a441a3a87e6fc149'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:17.664742 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 871/1753] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 03:35:17 2022] DELETE /identity/v3/users/f48848b5523c41c1a441a3a87e6fc149 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:35:17.669570 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cfde3b-22f3-4986-92c3-907fc33a4835 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:17.670057 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cfde3b-22f3-4986-92c3-907fc33a4835 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:17.670153 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cfde3b-22f3-4986-92c3-907fc33a4835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:17.689038 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-35cfde3b-22f3-4986-92c3-907fc33a4835 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:35:17.690216 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-35cfde3b-22f3-4986-92c3-907fc33a4835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:17.749300 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 883/1754] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:35:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:35:17.758900 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbfb5b9-ad26-43ca-8b4f-6c4474af94d6 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:17.760347 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-dcbfb5b9-ad26-43ca-8b4f-6c4474af94d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:17.818889 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbfb5b9-ad26-43ca-8b4f-6c4474af94d6 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:17.823877 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912228a4-fa8b-44fa-bd01-f6eb13bbc528 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:17.824533 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912228a4-fa8b-44fa-bd01-f6eb13bbc528 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:17.824807 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912228a4-fa8b-44fa-bd01-f6eb13bbc528 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:17.825134 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912228a4-fa8b-44fa-bd01-f6eb13bbc528 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:17.834624 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912228a4-fa8b-44fa-bd01-f6eb13bbc528 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:17.836060 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912228a4-fa8b-44fa-bd01-f6eb13bbc528 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:17.845843 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 872/1755] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:35:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:35:18.111318 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cfde3b-22f3-4986-92c3-907fc33a4835 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:18.112787 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-35cfde3b-22f3-4986-92c3-907fc33a4835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:18.127450 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cfde3b-22f3-4986-92c3-907fc33a4835 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:18.132521 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19037833-0975-4051-892c-f2ff7b28bf92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:18.133167 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19037833-0975-4051-892c-f2ff7b28bf92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:18.133603 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19037833-0975-4051-892c-f2ff7b28bf92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:18.133882 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19037833-0975-4051-892c-f2ff7b28bf92 admin admin] PATH_INFO: `/v3/projects/7f8277f6dfaf4d6bb5e2ece4d9b54b3b` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:18.135067 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19037833-0975-4051-892c-f2ff7b28bf92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f8277f6dfaf4d6bb5e2ece4d9b54b3b)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:18.136996 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19037833-0975-4051-892c-f2ff7b28bf92 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:18.188075 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-19037833-0975-4051-892c-f2ff7b28bf92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f8277f6dfaf4d6bb5e2ece4d9b54b3b', 'request_id': 'req-19037833-0975-4051-892c-f2ff7b28bf92'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:18.198172 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-19037833-0975-4051-892c-f2ff7b28bf92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f8277f6dfaf4d6bb5e2ece4d9b54b3b', 'request_id': 'req-19037833-0975-4051-892c-f2ff7b28bf92'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:18.201003 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:35:18.205840 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-19037833-0975-4051-892c-f2ff7b28bf92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f8277f6dfaf4d6bb5e2ece4d9b54b3b', 'request_id': 'req-19037833-0975-4051-892c-f2ff7b28bf92'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:18.220964 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-19037833-0975-4051-892c-f2ff7b28bf92 admin admin] The token cache is being invalidate because project 7f8277f6dfaf4d6bb5e2ece4d9b54b3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 03:35:18.221254 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-19037833-0975-4051-892c-f2ff7b28bf92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:18.222008 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-19037833-0975-4051-892c-f2ff7b28bf92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:18.223610 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 884/1756] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 03:35:18 2022] DELETE /identity/v3/projects/7f8277f6dfaf4d6bb5e2ece4d9b54b3b => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:35:18.228497 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912228a4-fa8b-44fa-bd01-f6eb13bbc528 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:18.238771 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912228a4-fa8b-44fa-bd01-f6eb13bbc528 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:18.316343 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a890e6-405e-4520-a284-c56bbac74c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:18.317100 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a890e6-405e-4520-a284-c56bbac74c2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:18.317280 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a890e6-405e-4520-a284-c56bbac74c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:18.317562 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a890e6-405e-4520-a284-c56bbac74c2a admin admin] PATH_INFO: `/v3/users/17d1195cbcb4455884810966956b2dd0` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:18.318702 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a890e6-405e-4520-a284-c56bbac74c2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=17d1195cbcb4455884810966956b2dd0)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:18.320677 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a890e6-405e-4520-a284-c56bbac74c2a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:18.384697 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b1a890e6-405e-4520-a284-c56bbac74c2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17d1195cbcb4455884810966956b2dd0'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:18.393787 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b1a890e6-405e-4520-a284-c56bbac74c2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17d1195cbcb4455884810966956b2dd0'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:18.394656 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b1a890e6-405e-4520-a284-c56bbac74c2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17d1195cbcb4455884810966956b2dd0'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:18.408253 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b1a890e6-405e-4520-a284-c56bbac74c2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17d1195cbcb4455884810966956b2dd0'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:18.417601 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b1a890e6-405e-4520-a284-c56bbac74c2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17d1195cbcb4455884810966956b2dd0'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:18.424563 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 873/1757] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 03:35:18 2022] DELETE /identity/v3/users/17d1195cbcb4455884810966956b2dd0 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:35:18.771343 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19037833-0975-4051-892c-f2ff7b28bf92 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:18.772991 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-19037833-0975-4051-892c-f2ff7b28bf92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:18.794136 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19037833-0975-4051-892c-f2ff7b28bf92 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:18.867053 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1446ec24-5f0d-475a-acf2-007aa4c184ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:18.867703 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446ec24-5f0d-475a-acf2-007aa4c184ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:18.868002 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446ec24-5f0d-475a-acf2-007aa4c184ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:18.868420 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1446ec24-5f0d-475a-acf2-007aa4c184ec admin admin] PATH_INFO: `/v3/projects/c15b76e29c2e4bec99b8e00d6ee0ec57` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:18.869609 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1446ec24-5f0d-475a-acf2-007aa4c184ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=c15b76e29c2e4bec99b8e00d6ee0ec57)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:18.871570 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1446ec24-5f0d-475a-acf2-007aa4c184ec admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:18.924683 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1446ec24-5f0d-475a-acf2-007aa4c184ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c15b76e29c2e4bec99b8e00d6ee0ec57', 'request_id': 'req-1446ec24-5f0d-475a-acf2-007aa4c184ec'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:18.931392 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1446ec24-5f0d-475a-acf2-007aa4c184ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c15b76e29c2e4bec99b8e00d6ee0ec57', 'request_id': 'req-1446ec24-5f0d-475a-acf2-007aa4c184ec'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:18.937627 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1446ec24-5f0d-475a-acf2-007aa4c184ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c15b76e29c2e4bec99b8e00d6ee0ec57', 'request_id': 'req-1446ec24-5f0d-475a-acf2-007aa4c184ec'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:18.950404 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1446ec24-5f0d-475a-acf2-007aa4c184ec admin admin] The token cache is being invalidate because project c15b76e29c2e4bec99b8e00d6ee0ec57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 03:35:18.950700 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1446ec24-5f0d-475a-acf2-007aa4c184ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:18.951332 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1446ec24-5f0d-475a-acf2-007aa4c184ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:18.953426 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 885/1758] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 03:35:18 2022] DELETE /identity/v3/projects/c15b76e29c2e4bec99b8e00d6ee0ec57 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:35:18.959196 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa07a9-e444-4cfb-bba2-4cddb9506b04 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:18.959683 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa07a9-e444-4cfb-bba2-4cddb9506b04 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:18.959933 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fa07a9-e444-4cfb-bba2-4cddb9506b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:18.976083 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:35:18.983739 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-05fa07a9-e444-4cfb-bba2-4cddb9506b04 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:35:18.985988 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-05fa07a9-e444-4cfb-bba2-4cddb9506b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:19.057024 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 874/1759] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:35:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:35:19.061540 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0dd318-8a54-40d4-8e66-9e158ca915f3 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:19.062459 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0dd318-8a54-40d4-8e66-9e158ca915f3 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:19.062735 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0dd318-8a54-40d4-8e66-9e158ca915f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:19.083098 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-de0dd318-8a54-40d4-8e66-9e158ca915f3 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:35:19.084326 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-de0dd318-8a54-40d4-8e66-9e158ca915f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:19.092728 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 886/1760] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:35:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:35:19.097020 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fa07a9-e444-4cfb-bba2-4cddb9506b04 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:19.107108 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fa07a9-e444-4cfb-bba2-4cddb9506b04 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:19.112247 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd667937-b9a4-41e4-a770-3b4124a4215d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:19.112855 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd667937-b9a4-41e4-a770-3b4124a4215d admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:19.113129 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd667937-b9a4-41e4-a770-3b4124a4215d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:19.113395 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd667937-b9a4-41e4-a770-3b4124a4215d admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:19.114036 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd667937-b9a4-41e4-a770-3b4124a4215d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:35:19.114332 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd667937-b9a4-41e4-a770-3b4124a4215d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:19.115971 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd667937-b9a4-41e4-a770-3b4124a4215d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:19.129119 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 875/1761] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 03:35:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:35:19.133548 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0dd318-8a54-40d4-8e66-9e158ca915f3 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:19.142848 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0dd318-8a54-40d4-8e66-9e158ca915f3 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:19.147555 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f6993e-2599-47cb-9d15-ab2a165502fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:19.148191 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f6993e-2599-47cb-9d15-ab2a165502fb admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:19.148474 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f6993e-2599-47cb-9d15-ab2a165502fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:19.148748 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f6993e-2599-47cb-9d15-ab2a165502fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:19.149381 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f6993e-2599-47cb-9d15-ab2a165502fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:19.151295 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f6993e-2599-47cb-9d15-ab2a165502fb admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:19.181582 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 887/1762] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:35:19 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:35:19.185743 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd667937-b9a4-41e4-a770-3b4124a4215d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:19.196658 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd667937-b9a4-41e4-a770-3b4124a4215d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:19.254832 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090aee6e-39c9-48a5-827e-8740399609cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:19.255694 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090aee6e-39c9-48a5-827e-8740399609cd admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:19.255694 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090aee6e-39c9-48a5-827e-8740399609cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:19.256031 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090aee6e-39c9-48a5-827e-8740399609cd admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:19.256650 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090aee6e-39c9-48a5-827e-8740399609cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:19.258590 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090aee6e-39c9-48a5-827e-8740399609cd admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:19.281120 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 876/1763] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:35:19 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:35:19.286012 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f6993e-2599-47cb-9d15-ab2a165502fb admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:19.299971 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f6993e-2599-47cb-9d15-ab2a165502fb admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:19.305684 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45ba7cd-7656-4093-9416-f9e9ef4c6950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:19.306319 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ba7cd-7656-4093-9416-f9e9ef4c6950 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:19.306627 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ba7cd-7656-4093-9416-f9e9ef4c6950 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:19.306875 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45ba7cd-7656-4093-9416-f9e9ef4c6950 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:19.307886 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45ba7cd-7656-4093-9416-f9e9ef4c6950 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:19.309598 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45ba7cd-7656-4093-9416-f9e9ef4c6950 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:19.324291 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 888/1764] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:35:19 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:35:19.325419 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090aee6e-39c9-48a5-827e-8740399609cd admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:19.337034 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090aee6e-39c9-48a5-827e-8740399609cd admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:19.343412 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abec1221-829c-4153-9b1d-9f4550bf7cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:19.344283 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abec1221-829c-4153-9b1d-9f4550bf7cb7 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:19.344593 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abec1221-829c-4153-9b1d-9f4550bf7cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:19.344992 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abec1221-829c-4153-9b1d-9f4550bf7cb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:19.345647 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abec1221-829c-4153-9b1d-9f4550bf7cb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:19.347778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abec1221-829c-4153-9b1d-9f4550bf7cb7 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:19.359792 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 877/1765] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:35:19 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:35:19.364346 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45ba7cd-7656-4093-9416-f9e9ef4c6950 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:19.376508 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45ba7cd-7656-4093-9416-f9e9ef4c6950 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:19.385698 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551399fb-2a6e-472f-aa56-b1a961572518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:19.386765 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551399fb-2a6e-472f-aa56-b1a961572518 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:19.387141 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551399fb-2a6e-472f-aa56-b1a961572518 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:19.387516 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551399fb-2a6e-472f-aa56-b1a961572518 admin admin] PATH_INFO: `/v3/projects/0b3a0a3d2512405b869c4f7f70927064/users/feb3d7c64d5247a58c9e1e606c62b476/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:19.412202 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551399fb-2a6e-472f-aa56-b1a961572518 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b3a0a3d2512405b869c4f7f70927064, user_id=feb3d7c64d5247a58c9e1e606c62b476, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:19.415732 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551399fb-2a6e-472f-aa56-b1a961572518 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:19.431432 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 889/1766] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:35:19 2022] PUT /identity/v3/projects/0b3a0a3d2512405b869c4f7f70927064/users/feb3d7c64d5247a58c9e1e606c62b476/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:35:19.445618 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4559ef36-e6e1-4c1f-b2c7-b96705a5a57d None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:19.445909 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4559ef36-e6e1-4c1f-b2c7-b96705a5a57d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:19.446218 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4559ef36-e6e1-4c1f-b2c7-b96705a5a57d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:19.512623 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-4559ef36-e6e1-4c1f-b2c7-b96705a5a57d None None] MFA Rules not processed for user `feb3d7c64d5247a58c9e1e606c62b476`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:35:19.513729 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4559ef36-e6e1-4c1f-b2c7-b96705a5a57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:19.593356 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 878/1767] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:35:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:35:19.595698 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551399fb-2a6e-472f-aa56-b1a961572518 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:19.615765 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:35:19.682768 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551399fb-2a6e-472f-aa56-b1a961572518 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:19.689769 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1830beba-7605-4ee7-8dc2-8b620b70d7d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:19.692040 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1830beba-7605-4ee7-8dc2-8b620b70d7d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:19.692316 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1830beba-7605-4ee7-8dc2-8b620b70d7d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:19.692595 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1830beba-7605-4ee7-8dc2-8b620b70d7d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:19.693261 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1830beba-7605-4ee7-8dc2-8b620b70d7d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:19.698755 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1830beba-7605-4ee7-8dc2-8b620b70d7d4 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:19.699818 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4559ef36-e6e1-4c1f-b2c7-b96705a5a57d None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:19.701593 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4559ef36-e6e1-4c1f-b2c7-b96705a5a57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:19.713520 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:35:19.724131 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4559ef36-e6e1-4c1f-b2c7-b96705a5a57d None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:19.731259 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813b05ba-654d-43b4-a3a8-910b3e90daf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:19.731925 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813b05ba-654d-43b4-a3a8-910b3e90daf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:19.732240 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813b05ba-654d-43b4-a3a8-910b3e90daf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:19.732672 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813b05ba-654d-43b4-a3a8-910b3e90daf7 admin admin] PATH_INFO: `/v3/users/725a3258d5ab4c098e29e3d9a5c5959a` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:19.734372 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813b05ba-654d-43b4-a3a8-910b3e90daf7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=725a3258d5ab4c098e29e3d9a5c5959a)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:19.736380 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813b05ba-654d-43b4-a3a8-910b3e90daf7 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:19.740009 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 890/1768] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:35:19 2022] POST /identity/v3/projects => generated 375 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:35:19.741870 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1830beba-7605-4ee7-8dc2-8b620b70d7d4 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:19.800569 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-813b05ba-654d-43b4-a3a8-910b3e90daf7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '725a3258d5ab4c098e29e3d9a5c5959a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:19.808441 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-813b05ba-654d-43b4-a3a8-910b3e90daf7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '725a3258d5ab4c098e29e3d9a5c5959a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:19.809286 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-813b05ba-654d-43b4-a3a8-910b3e90daf7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '725a3258d5ab4c098e29e3d9a5c5959a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:19.819835 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1830beba-7605-4ee7-8dc2-8b620b70d7d4 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:19.823400 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-813b05ba-654d-43b4-a3a8-910b3e90daf7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '725a3258d5ab4c098e29e3d9a5c5959a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:19.825414 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c931b5b-4e66-4a02-8908-4a67abbb7857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:19.826153 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c931b5b-4e66-4a02-8908-4a67abbb7857 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:19.826503 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c931b5b-4e66-4a02-8908-4a67abbb7857 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:19.826784 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c931b5b-4e66-4a02-8908-4a67abbb7857 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:19.828213 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c931b5b-4e66-4a02-8908-4a67abbb7857 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:19.830354 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c931b5b-4e66-4a02-8908-4a67abbb7857 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:19.833742 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-813b05ba-654d-43b4-a3a8-910b3e90daf7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '725a3258d5ab4c098e29e3d9a5c5959a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:19.842851 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 879/1769] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 03:35:19 2022] DELETE /identity/v3/users/725a3258d5ab4c098e29e3d9a5c5959a => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 03:35:19.847669 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def21e8b-9c55-4570-9ff3-a9fba1420155 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:19.847973 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def21e8b-9c55-4570-9ff3-a9fba1420155 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:19.848282 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def21e8b-9c55-4570-9ff3-a9fba1420155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:19.857056 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 891/1770] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:35:19 2022] POST /identity/v3/users => generated 288 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 03:35:19.861478 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c931b5b-4e66-4a02-8908-4a67abbb7857 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:19.865426 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-0c931b5b-4e66-4a02-8908-4a67abbb7857 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:19.868636 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-def21e8b-9c55-4570-9ff3-a9fba1420155 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:35:19.869925 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-def21e8b-9c55-4570-9ff3-a9fba1420155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:19.879166 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c931b5b-4e66-4a02-8908-4a67abbb7857 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:19.945091 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 880/1771] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:35:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:35:19.949204 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84bb4a9-a24c-4f24-80d9-cc1c9470cecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:19.949718 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84bb4a9-a24c-4f24-80d9-cc1c9470cecd admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:19.949995 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84bb4a9-a24c-4f24-80d9-cc1c9470cecd admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:19.950269 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84bb4a9-a24c-4f24-80d9-cc1c9470cecd admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:19.950943 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84bb4a9-a24c-4f24-80d9-cc1c9470cecd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:19.953833 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84bb4a9-a24c-4f24-80d9-cc1c9470cecd admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:19.958045 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def21e8b-9c55-4570-9ff3-a9fba1420155 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:19.959522 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-def21e8b-9c55-4570-9ff3-a9fba1420155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:19.964652 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 892/1772] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:35:19 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 03:35:19.968687 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84bb4a9-a24c-4f24-80d9-cc1c9470cecd admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:19.979206 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84bb4a9-a24c-4f24-80d9-cc1c9470cecd admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:19.996160 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c511b5d5-d700-431b-9195-e34bed51aaaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:19.996160 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c511b5d5-d700-431b-9195-e34bed51aaaf admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:19.996160 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c511b5d5-d700-431b-9195-e34bed51aaaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:19.996160 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c511b5d5-d700-431b-9195-e34bed51aaaf admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:19.996536 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c511b5d5-d700-431b-9195-e34bed51aaaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:19.996536 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c511b5d5-d700-431b-9195-e34bed51aaaf admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:20.000243 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 893/1773] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:35:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 03:35:20.004541 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c511b5d5-d700-431b-9195-e34bed51aaaf admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:20.016632 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c511b5d5-d700-431b-9195-e34bed51aaaf admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:20.021194 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c864b08a-0009-420c-b645-55a39b0a3347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:20.021834 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c864b08a-0009-420c-b645-55a39b0a3347 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:20.022111 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c864b08a-0009-420c-b645-55a39b0a3347 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:20.022917 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c864b08a-0009-420c-b645-55a39b0a3347 admin admin] PATH_INFO: `/v3/projects/617a422546074a37b8f5609195512bef/users/f6e5b241d57348bea96ed8bb7cf9d6c0/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:20.026161 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def21e8b-9c55-4570-9ff3-a9fba1420155 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:20.030047 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-139ce225-b3db-4563-8b4c-50633b02a15e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:20.031058 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ce225-b3db-4563-8b4c-50633b02a15e service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:20.031058 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ce225-b3db-4563-8b4c-50633b02a15e service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:20.031270 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-139ce225-b3db-4563-8b4c-50633b02a15e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:20.034742 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:35:20.038187 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c864b08a-0009-420c-b645-55a39b0a3347 admin admin] RBAC: Authorizing `identity:create_grant(project_id=617a422546074a37b8f5609195512bef, user_id=f6e5b241d57348bea96ed8bb7cf9d6c0, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:20.040841 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139ce225-b3db-4563-8b4c-50633b02a15e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:20.041495 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c864b08a-0009-420c-b645-55a39b0a3347 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:20.042465 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-139ce225-b3db-4563-8b4c-50633b02a15e service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:20.052517 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 881/1774] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:35:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:35:20.054828 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 894/1775] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:35:20 2022] PUT /identity/v3/projects/617a422546074a37b8f5609195512bef/users/f6e5b241d57348bea96ed8bb7cf9d6c0/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 03:35:20.066000 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fc6ae3-e237-47bf-8809-41488f66f582 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:20.066355 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fc6ae3-e237-47bf-8809-41488f66f582 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:20.066726 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fc6ae3-e237-47bf-8809-41488f66f582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:20.098581 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-46fc6ae3-e237-47bf-8809-41488f66f582 None None] MFA Rules not processed for user `f6e5b241d57348bea96ed8bb7cf9d6c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:35:20.099804 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-46fc6ae3-e237-47bf-8809-41488f66f582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:20.105808 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:35:20.175652 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 882/1776] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:35:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:35:20.180182 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c864b08a-0009-420c-b645-55a39b0a3347 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:20.251069 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c864b08a-0009-420c-b645-55a39b0a3347 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:20.256149 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26376cf3-0618-473c-99fc-73ebd0d4c723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:20.256766 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26376cf3-0618-473c-99fc-73ebd0d4c723 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:20.257049 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26376cf3-0618-473c-99fc-73ebd0d4c723 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:20.257325 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26376cf3-0618-473c-99fc-73ebd0d4c723 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:20.257974 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26376cf3-0618-473c-99fc-73ebd0d4c723 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:20.259884 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26376cf3-0618-473c-99fc-73ebd0d4c723 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:20.292244 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 895/1777] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:35:20 2022] POST /identity/v3/projects => generated 377 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:35:20.293426 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fc6ae3-e237-47bf-8809-41488f66f582 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:20.304951 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fc6ae3-e237-47bf-8809-41488f66f582 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:20.348454 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26376cf3-0618-473c-99fc-73ebd0d4c723 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:20.351131 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-26376cf3-0618-473c-99fc-73ebd0d4c723 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:20.368365 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b82924-3890-4abc-b534-d709577bcff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:20.368989 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b82924-3890-4abc-b534-d709577bcff9 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:20.369264 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b82924-3890-4abc-b534-d709577bcff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:20.369532 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b82924-3890-4abc-b534-d709577bcff9 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:20.370166 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b82924-3890-4abc-b534-d709577bcff9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:20.373058 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b82924-3890-4abc-b534-d709577bcff9 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:20.403710 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 883/1778] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:35:20 2022] POST /identity/v3/users => generated 295 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:35:20.407689 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b82924-3890-4abc-b534-d709577bcff9 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:20.417422 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b82924-3890-4abc-b534-d709577bcff9 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:20.422878 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a6ec1-1fb5-424c-806c-320151122dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:20.423897 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a6ec1-1fb5-424c-806c-320151122dad admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:20.424206 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a6ec1-1fb5-424c-806c-320151122dad admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:20.424479 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a6ec1-1fb5-424c-806c-320151122dad admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:20.425117 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109a6ec1-1fb5-424c-806c-320151122dad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:20.426731 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109a6ec1-1fb5-424c-806c-320151122dad admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:20.427734 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26376cf3-0618-473c-99fc-73ebd0d4c723 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:20.432862 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571136bd-59e7-4ec4-bf9d-428b6fb2ddc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:20.433817 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571136bd-59e7-4ec4-bf9d-428b6fb2ddc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:20.434144 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571136bd-59e7-4ec4-bf9d-428b6fb2ddc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:20.434453 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571136bd-59e7-4ec4-bf9d-428b6fb2ddc4 admin admin] PATH_INFO: `/v3/projects/ee471c51adf245578fd44ac37791dbe9` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:20.435704 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571136bd-59e7-4ec4-bf9d-428b6fb2ddc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee471c51adf245578fd44ac37791dbe9)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:20.438247 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571136bd-59e7-4ec4-bf9d-428b6fb2ddc4 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:20.439361 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 884/1779] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:35:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 03:35:20.443428 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a6ec1-1fb5-424c-806c-320151122dad admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:20.453493 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a6ec1-1fb5-424c-806c-320151122dad admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:20.458279 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f141c9f7-a88c-4059-8440-232c5f78c63d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:20.458990 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f141c9f7-a88c-4059-8440-232c5f78c63d admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:20.459269 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f141c9f7-a88c-4059-8440-232c5f78c63d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:20.459539 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f141c9f7-a88c-4059-8440-232c5f78c63d admin admin] PATH_INFO: `/v3/projects/22fa95e6a9614b6ba1d969f5b008bd16/users/af798437fbbd4793985f43fc4199ad51/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:20.475376 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f141c9f7-a88c-4059-8440-232c5f78c63d admin admin] RBAC: Authorizing `identity:create_grant(project_id=22fa95e6a9614b6ba1d969f5b008bd16, user_id=af798437fbbd4793985f43fc4199ad51, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:20.478085 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f141c9f7-a88c-4059-8440-232c5f78c63d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:20.489588 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-571136bd-59e7-4ec4-bf9d-428b6fb2ddc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee471c51adf245578fd44ac37791dbe9', 'request_id': 'req-571136bd-59e7-4ec4-bf9d-428b6fb2ddc4'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:20.490910 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 885/1780] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:35:20 2022] PUT /identity/v3/projects/22fa95e6a9614b6ba1d969f5b008bd16/users/af798437fbbd4793985f43fc4199ad51/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 03:35:20.494747 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f141c9f7-a88c-4059-8440-232c5f78c63d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:20.495518 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-571136bd-59e7-4ec4-bf9d-428b6fb2ddc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee471c51adf245578fd44ac37791dbe9', 'request_id': 'req-571136bd-59e7-4ec4-bf9d-428b6fb2ddc4'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:20.504582 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-571136bd-59e7-4ec4-bf9d-428b6fb2ddc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee471c51adf245578fd44ac37791dbe9', 'request_id': 'req-571136bd-59e7-4ec4-bf9d-428b6fb2ddc4'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:20.519573 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-571136bd-59e7-4ec4-bf9d-428b6fb2ddc4 admin admin] The token cache is being invalidate because project ee471c51adf245578fd44ac37791dbe9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 03:35:20.520149 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-571136bd-59e7-4ec4-bf9d-428b6fb2ddc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:20.520857 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-571136bd-59e7-4ec4-bf9d-428b6fb2ddc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:20.522389 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 896/1781] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 03:35:20 2022] DELETE /identity/v3/projects/ee471c51adf245578fd44ac37791dbe9 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 03:35:20.526322 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571136bd-59e7-4ec4-bf9d-428b6fb2ddc4 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:20.527723 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-571136bd-59e7-4ec4-bf9d-428b6fb2ddc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:20.538281 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:35:20.575651 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f141c9f7-a88c-4059-8440-232c5f78c63d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:20.581115 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bfabe7-f088-41bc-b010-c2c165b081c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:20.581923 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bfabe7-f088-41bc-b010-c2c165b081c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:20.582488 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bfabe7-f088-41bc-b010-c2c165b081c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:20.582488 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bfabe7-f088-41bc-b010-c2c165b081c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:20.583116 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bfabe7-f088-41bc-b010-c2c165b081c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:20.584776 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1bfabe7-f088-41bc-b010-c2c165b081c8 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:20.587499 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:35:20.602610 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 886/1782] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:35:20 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 03:35:20.603497 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bfabe7-f088-41bc-b010-c2c165b081c8 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:20.605502 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f1bfabe7-f088-41bc-b010-c2c165b081c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:20.609545 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571136bd-59e7-4ec4-bf9d-428b6fb2ddc4 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:20.615998 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23801499-4f15-4fe3-bc6f-6184d11f0809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:20.615998 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23801499-4f15-4fe3-bc6f-6184d11f0809 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:20.615998 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23801499-4f15-4fe3-bc6f-6184d11f0809 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:20.616722 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23801499-4f15-4fe3-bc6f-6184d11f0809 admin admin] PATH_INFO: `/v3/users/3b088b3ef4df45a8aeadd25953b98f05` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:20.620540 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23801499-4f15-4fe3-bc6f-6184d11f0809 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b088b3ef4df45a8aeadd25953b98f05)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:20.622523 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23801499-4f15-4fe3-bc6f-6184d11f0809 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:20.626361 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bfabe7-f088-41bc-b010-c2c165b081c8 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:20.631719 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c72b4-88be-410d-913a-8d512c44ae8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:20.632432 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c72b4-88be-410d-913a-8d512c44ae8d admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:20.632715 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c72b4-88be-410d-913a-8d512c44ae8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:20.632995 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c72b4-88be-410d-913a-8d512c44ae8d admin admin] PATH_INFO: `/v3/projects/22fa95e6a9614b6ba1d969f5b008bd16/users/af798437fbbd4793985f43fc4199ad51/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:20.635030 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941c72b4-88be-410d-913a-8d512c44ae8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=22fa95e6a9614b6ba1d969f5b008bd16, user_id=af798437fbbd4793985f43fc4199ad51, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:20.637701 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941c72b4-88be-410d-913a-8d512c44ae8d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:20.651445 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 887/1783] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:35:20 2022] PUT /identity/v3/projects/22fa95e6a9614b6ba1d969f5b008bd16/users/af798437fbbd4793985f43fc4199ad51/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:35:20.661467 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d3542-011f-4040-8fa3-b1eee83f01ed None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:20.661864 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d3542-011f-4040-8fa3-b1eee83f01ed None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:20.662156 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d3542-011f-4040-8fa3-b1eee83f01ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:20.678101 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-23801499-4f15-4fe3-bc6f-6184d11f0809 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b088b3ef4df45a8aeadd25953b98f05'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:20.678778 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-23801499-4f15-4fe3-bc6f-6184d11f0809 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b088b3ef4df45a8aeadd25953b98f05'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:20.679471 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-23801499-4f15-4fe3-bc6f-6184d11f0809 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b088b3ef4df45a8aeadd25953b98f05'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:20.692888 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-750d3542-011f-4040-8fa3-b1eee83f01ed None None] MFA Rules not processed for user `af798437fbbd4793985f43fc4199ad51`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:35:20.694121 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-750d3542-011f-4040-8fa3-b1eee83f01ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:20.697741 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-23801499-4f15-4fe3-bc6f-6184d11f0809 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b088b3ef4df45a8aeadd25953b98f05'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:20.705556 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-23801499-4f15-4fe3-bc6f-6184d11f0809 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b088b3ef4df45a8aeadd25953b98f05'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:20.720991 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 897/1784] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 03:35:20 2022] DELETE /identity/v3/users/3b088b3ef4df45a8aeadd25953b98f05 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 03:35:20.778938 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 888/1785] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:35:20 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 03:35:20.791449 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23801499-4f15-4fe3-bc6f-6184d11f0809 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:20.793047 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-23801499-4f15-4fe3-bc6f-6184d11f0809 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:20.857848 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23801499-4f15-4fe3-bc6f-6184d11f0809 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:20.861619 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d5cce2-7ae1-4cac-993e-d076f9a05d84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:20.862228 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d5cce2-7ae1-4cac-993e-d076f9a05d84 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:20.862508 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d5cce2-7ae1-4cac-993e-d076f9a05d84 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:20.862781 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d5cce2-7ae1-4cac-993e-d076f9a05d84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:20.864625 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-44d5cce2-7ae1-4cac-993e-d076f9a05d84 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:20.918353 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:35:20.938863 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d5cce2-7ae1-4cac-993e-d076f9a05d84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:20.940413 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d5cce2-7ae1-4cac-993e-d076f9a05d84 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:20.949439 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 898/1786] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:35:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:35:21.018656 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750d3542-011f-4040-8fa3-b1eee83f01ed None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:21.020404 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-750d3542-011f-4040-8fa3-b1eee83f01ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:21.127100 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750d3542-011f-4040-8fa3-b1eee83f01ed None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:21.191761 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775d5852-25b2-47cb-9621-238689450b20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:21.193385 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775d5852-25b2-47cb-9621-238689450b20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:21.193820 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775d5852-25b2-47cb-9621-238689450b20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:21.194257 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775d5852-25b2-47cb-9621-238689450b20 admin admin] PATH_INFO: `/v3/projects/de15164e07c24535a93ef2f4b341760e` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:21.196211 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775d5852-25b2-47cb-9621-238689450b20 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de15164e07c24535a93ef2f4b341760e)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:21.198649 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775d5852-25b2-47cb-9621-238689450b20 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:21.225368 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:35:21.254332 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-775d5852-25b2-47cb-9621-238689450b20 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de15164e07c24535a93ef2f4b341760e', 'request_id': 'req-775d5852-25b2-47cb-9621-238689450b20'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:21.261893 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-775d5852-25b2-47cb-9621-238689450b20 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de15164e07c24535a93ef2f4b341760e', 'request_id': 'req-775d5852-25b2-47cb-9621-238689450b20'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:21.270679 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-775d5852-25b2-47cb-9621-238689450b20 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de15164e07c24535a93ef2f4b341760e', 'request_id': 'req-775d5852-25b2-47cb-9621-238689450b20'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:21.284475 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-775d5852-25b2-47cb-9621-238689450b20 admin admin] The token cache is being invalidate because project de15164e07c24535a93ef2f4b341760e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 03:35:21.284475 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-775d5852-25b2-47cb-9621-238689450b20 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:21.285004 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-775d5852-25b2-47cb-9621-238689450b20 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:21.287374 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 889/1787] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 03:35:21 2022] DELETE /identity/v3/projects/de15164e07c24535a93ef2f4b341760e => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:35:21.295934 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a17dc61-7b11-432b-8c4d-de936720f62c None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:21.296991 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a17dc61-7b11-432b-8c4d-de936720f62c None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:21.297299 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a17dc61-7b11-432b-8c4d-de936720f62c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:21.321889 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-8a17dc61-7b11-432b-8c4d-de936720f62c None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:35:21.323182 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-8a17dc61-7b11-432b-8c4d-de936720f62c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:21.395451 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 899/1788] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:35:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:35:21.403324 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3690e1b-fd16-45fa-a11c-9eba162ec0db None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:21.403615 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3690e1b-fd16-45fa-a11c-9eba162ec0db None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:21.403888 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3690e1b-fd16-45fa-a11c-9eba162ec0db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:21.425028 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d3690e1b-fd16-45fa-a11c-9eba162ec0db None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:35:21.426251 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d3690e1b-fd16-45fa-a11c-9eba162ec0db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:21.434528 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 890/1789] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:35:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:35:21.439211 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a17dc61-7b11-432b-8c4d-de936720f62c None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:21.448830 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a17dc61-7b11-432b-8c4d-de936720f62c None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:21.454249 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ad25b2-a808-4719-8c9b-ae447ac324c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:21.454857 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad25b2-a808-4719-8c9b-ae447ac324c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:21.455183 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad25b2-a808-4719-8c9b-ae447ac324c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:21.455457 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad25b2-a808-4719-8c9b-ae447ac324c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:21.456144 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ad25b2-a808-4719-8c9b-ae447ac324c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:35:21.456454 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ad25b2-a808-4719-8c9b-ae447ac324c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:21.458111 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ad25b2-a808-4719-8c9b-ae447ac324c8 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:21.469690 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 900/1790] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 03:35:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:35:21.473810 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3690e1b-fd16-45fa-a11c-9eba162ec0db None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:21.483135 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3690e1b-fd16-45fa-a11c-9eba162ec0db None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:21.487776 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15879261-e6db-4065-9eda-be8eba2c3898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:21.488411 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15879261-e6db-4065-9eda-be8eba2c3898 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:21.488695 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15879261-e6db-4065-9eda-be8eba2c3898 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:21.488972 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15879261-e6db-4065-9eda-be8eba2c3898 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:21.489628 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15879261-e6db-4065-9eda-be8eba2c3898 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:21.491515 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15879261-e6db-4065-9eda-be8eba2c3898 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:21.518815 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 891/1791] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:35:21 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:35:21.522878 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ad25b2-a808-4719-8c9b-ae447ac324c8 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:21.533073 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ad25b2-a808-4719-8c9b-ae447ac324c8 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:21.588789 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c6971e-3d6d-47d9-826f-0c5e9a48a743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:21.589393 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c6971e-3d6d-47d9-826f-0c5e9a48a743 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:21.589672 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c6971e-3d6d-47d9-826f-0c5e9a48a743 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:21.589987 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c6971e-3d6d-47d9-826f-0c5e9a48a743 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:21.590558 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c6971e-3d6d-47d9-826f-0c5e9a48a743 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:21.592482 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c6971e-3d6d-47d9-826f-0c5e9a48a743 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:21.613349 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 901/1792] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:35:21 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:35:21.617336 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15879261-e6db-4065-9eda-be8eba2c3898 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:21.628707 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15879261-e6db-4065-9eda-be8eba2c3898 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:21.633458 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1314401-c451-435c-8958-23601675b832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:21.634061 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1314401-c451-435c-8958-23601675b832 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:21.634342 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1314401-c451-435c-8958-23601675b832 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:21.634616 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1314401-c451-435c-8958-23601675b832 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:21.635251 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1314401-c451-435c-8958-23601675b832 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:21.636889 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1314401-c451-435c-8958-23601675b832 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:21.647702 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 892/1793] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:35:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:35:21.651456 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c6971e-3d6d-47d9-826f-0c5e9a48a743 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:21.661315 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c6971e-3d6d-47d9-826f-0c5e9a48a743 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:21.665831 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da07f88e-d95d-410c-b856-cdb36b011f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:21.666420 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da07f88e-d95d-410c-b856-cdb36b011f03 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:21.666698 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da07f88e-d95d-410c-b856-cdb36b011f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:21.666967 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da07f88e-d95d-410c-b856-cdb36b011f03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:21.667596 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da07f88e-d95d-410c-b856-cdb36b011f03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:21.669282 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da07f88e-d95d-410c-b856-cdb36b011f03 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:21.680621 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 902/1794] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:35:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:35:21.684831 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1314401-c451-435c-8958-23601675b832 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:21.695347 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1314401-c451-435c-8958-23601675b832 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:21.699834 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1565ed-d4fb-49fc-aecf-9723ae548fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:21.700523 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1565ed-d4fb-49fc-aecf-9723ae548fb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:21.700800 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1565ed-d4fb-49fc-aecf-9723ae548fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:21.701188 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1565ed-d4fb-49fc-aecf-9723ae548fb1 admin admin] PATH_INFO: `/v3/projects/513566a027164bab9f21f49f96bb878d/users/737aa5b75c714fe7ad3a1229e9b19700/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:21.716028 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1565ed-d4fb-49fc-aecf-9723ae548fb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=513566a027164bab9f21f49f96bb878d, user_id=737aa5b75c714fe7ad3a1229e9b19700, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:21.718676 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1565ed-d4fb-49fc-aecf-9723ae548fb1 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:21.731801 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 893/1795] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:35:21 2022] PUT /identity/v3/projects/513566a027164bab9f21f49f96bb878d/users/737aa5b75c714fe7ad3a1229e9b19700/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:35:21.736848 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857fb4ab-6a93-42bd-a04c-5103044350e3 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:21.737137 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857fb4ab-6a93-42bd-a04c-5103044350e3 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:21.737412 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857fb4ab-6a93-42bd-a04c-5103044350e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:21.757438 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-857fb4ab-6a93-42bd-a04c-5103044350e3 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:35:21.758653 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-857fb4ab-6a93-42bd-a04c-5103044350e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:21.801442 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:35:21.823105 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 903/1796] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:35:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:35:21.833679 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1565ed-d4fb-49fc-aecf-9723ae548fb1 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:21.835406 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8a1565ed-d4fb-49fc-aecf-9723ae548fb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:21.894942 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1565ed-d4fb-49fc-aecf-9723ae548fb1 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:21.898487 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484dfb0e-0837-42b6-bec1-d047cdadf6cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:21.899100 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484dfb0e-0837-42b6-bec1-d047cdadf6cf service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:21.899382 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484dfb0e-0837-42b6-bec1-d047cdadf6cf service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:21.899654 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484dfb0e-0837-42b6-bec1-d047cdadf6cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:21.908999 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484dfb0e-0837-42b6-bec1-d047cdadf6cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:21.910446 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484dfb0e-0837-42b6-bec1-d047cdadf6cf service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:21.919793 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 894/1797] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:35:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:35:22.618145 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857fb4ab-6a93-42bd-a04c-5103044350e3 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:22.624766 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-857fb4ab-6a93-42bd-a04c-5103044350e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:22.697208 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857fb4ab-6a93-42bd-a04c-5103044350e3 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:22.703856 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458c5e76-ede8-4a19-b97d-1870a1ebb1f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:22.704692 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458c5e76-ede8-4a19-b97d-1870a1ebb1f5 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:22.704963 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458c5e76-ede8-4a19-b97d-1870a1ebb1f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:22.705283 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458c5e76-ede8-4a19-b97d-1870a1ebb1f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:22.707110 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-458c5e76-ede8-4a19-b97d-1870a1ebb1f5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:22.726530 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa0781e-6d80-4ade-9a09-503abcf9b4eb None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:22.727671 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa0781e-6d80-4ade-9a09-503abcf9b4eb None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:22.728525 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa0781e-6d80-4ade-9a09-503abcf9b4eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:22.762382 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-3fa0781e-6d80-4ade-9a09-503abcf9b4eb None None] MFA Rules not processed for user `737aa5b75c714fe7ad3a1229e9b19700`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:35:22.763815 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3fa0781e-6d80-4ade-9a09-503abcf9b4eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:22.769874 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458c5e76-ede8-4a19-b97d-1870a1ebb1f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:22.771310 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458c5e76-ede8-4a19-b97d-1870a1ebb1f5 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:22.783843 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 904/1798] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:35:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:35:22.816407 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:35:22.891098 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 895/1799] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:35:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 03:35:22.895772 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458c5e76-ede8-4a19-b97d-1870a1ebb1f5 service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:22.912590 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458c5e76-ede8-4a19-b97d-1870a1ebb1f5 service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:22.938134 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213b3de6-ac1f-449e-b08c-7f7fd25b6e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:22.939065 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b3de6-ac1f-449e-b08c-7f7fd25b6e55 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:22.939412 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b3de6-ac1f-449e-b08c-7f7fd25b6e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:22.939756 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213b3de6-ac1f-449e-b08c-7f7fd25b6e55 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:22.940486 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213b3de6-ac1f-449e-b08c-7f7fd25b6e55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:22.942477 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213b3de6-ac1f-449e-b08c-7f7fd25b6e55 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:22.985581 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 905/1800] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:35:22 2022] POST /identity/v3/projects => generated 373 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:35:22.989508 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa0781e-6d80-4ade-9a09-503abcf9b4eb None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:23.008174 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa0781e-6d80-4ade-9a09-503abcf9b4eb None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:23.083031 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c3e67d-3d49-4311-a391-2d76ef6b0529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:23.083665 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c3e67d-3d49-4311-a391-2d76ef6b0529 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:23.084317 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c3e67d-3d49-4311-a391-2d76ef6b0529 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:23.084680 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c3e67d-3d49-4311-a391-2d76ef6b0529 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:23.085715 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c3e67d-3d49-4311-a391-2d76ef6b0529 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:23.087714 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c3e67d-3d49-4311-a391-2d76ef6b0529 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:23.114930 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 896/1801] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:35:22 2022] POST /identity/v3/users => generated 293 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:35:23.117485 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213b3de6-ac1f-449e-b08c-7f7fd25b6e55 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:23.135094 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213b3de6-ac1f-449e-b08c-7f7fd25b6e55 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:23.140633 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e02726f-4288-456e-9bec-c5cded4f24eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:23.141652 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e02726f-4288-456e-9bec-c5cded4f24eb admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:23.142140 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e02726f-4288-456e-9bec-c5cded4f24eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:23.142604 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e02726f-4288-456e-9bec-c5cded4f24eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:23.143558 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e02726f-4288-456e-9bec-c5cded4f24eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:23.145510 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e02726f-4288-456e-9bec-c5cded4f24eb admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:23.163584 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 906/1802] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:35:23 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 03:35:23.165055 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c3e67d-3d49-4311-a391-2d76ef6b0529 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:23.175852 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c3e67d-3d49-4311-a391-2d76ef6b0529 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:23.180990 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e010656a-09d0-4a5d-9bf7-aef75d24a47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:23.181648 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e010656a-09d0-4a5d-9bf7-aef75d24a47d admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:23.181932 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e010656a-09d0-4a5d-9bf7-aef75d24a47d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:23.182209 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e010656a-09d0-4a5d-9bf7-aef75d24a47d admin admin] PATH_INFO: `/v3/projects/7fb66b88418141aca86a24340356dec6/users/b0df4a85f68d4cdf8e2d34ac4fbd3874/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:23.197916 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e010656a-09d0-4a5d-9bf7-aef75d24a47d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fb66b88418141aca86a24340356dec6, user_id=b0df4a85f68d4cdf8e2d34ac4fbd3874, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:23.200557 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e010656a-09d0-4a5d-9bf7-aef75d24a47d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:23.213844 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 897/1803] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:35:23 2022] PUT /identity/v3/projects/7fb66b88418141aca86a24340356dec6/users/b0df4a85f68d4cdf8e2d34ac4fbd3874/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:35:23.218333 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e02726f-4288-456e-9bec-c5cded4f24eb admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:23.228822 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e02726f-4288-456e-9bec-c5cded4f24eb admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:23.294989 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b21e0b8-ddc8-47ea-ad80-da75d70176df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:23.295669 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21e0b8-ddc8-47ea-ad80-da75d70176df admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:23.296031 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21e0b8-ddc8-47ea-ad80-da75d70176df admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:23.296828 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21e0b8-ddc8-47ea-ad80-da75d70176df admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:23.297481 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b21e0b8-ddc8-47ea-ad80-da75d70176df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:23.299064 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b21e0b8-ddc8-47ea-ad80-da75d70176df admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:23.313056 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 907/1804] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:35:23 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:35:23.321000 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e010656a-09d0-4a5d-9bf7-aef75d24a47d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:23.333336 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e010656a-09d0-4a5d-9bf7-aef75d24a47d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:23.338854 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5a6df0-98f1-4bd1-8a4c-19a64db626e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:23.339551 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a6df0-98f1-4bd1-8a4c-19a64db626e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:23.339819 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a6df0-98f1-4bd1-8a4c-19a64db626e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:23.340120 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5a6df0-98f1-4bd1-8a4c-19a64db626e2 admin admin] PATH_INFO: `/v3/projects/7fb66b88418141aca86a24340356dec6/users/b0df4a85f68d4cdf8e2d34ac4fbd3874/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:23.342172 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5a6df0-98f1-4bd1-8a4c-19a64db626e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fb66b88418141aca86a24340356dec6, user_id=b0df4a85f68d4cdf8e2d34ac4fbd3874, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:23.346030 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b5a6df0-98f1-4bd1-8a4c-19a64db626e2 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:23.358920 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 898/1805] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:35:23 2022] PUT /identity/v3/projects/7fb66b88418141aca86a24340356dec6/users/b0df4a85f68d4cdf8e2d34ac4fbd3874/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:35:23.571719 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b21e0b8-ddc8-47ea-ad80-da75d70176df admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:23.577404 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:35:23.629665 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b21e0b8-ddc8-47ea-ad80-da75d70176df admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:23.696085 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986cc4b4-cd35-48b5-b5cc-bfff63fa1d28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:23.696795 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986cc4b4-cd35-48b5-b5cc-bfff63fa1d28 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:23.697143 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986cc4b4-cd35-48b5-b5cc-bfff63fa1d28 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:23.697494 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986cc4b4-cd35-48b5-b5cc-bfff63fa1d28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:23.699384 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-986cc4b4-cd35-48b5-b5cc-bfff63fa1d28 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:23.830324 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986cc4b4-cd35-48b5-b5cc-bfff63fa1d28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:23.831907 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986cc4b4-cd35-48b5-b5cc-bfff63fa1d28 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:23.864590 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 908/1806] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:35:23 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:35:24.164658 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c340ece-3ab1-4aa8-85ac-b51abaa9871f None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:24.165203 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c340ece-3ab1-4aa8-85ac-b51abaa9871f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:24.165570 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c340ece-3ab1-4aa8-85ac-b51abaa9871f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:24.197778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-9c340ece-3ab1-4aa8-85ac-b51abaa9871f None None] MFA Rules not processed for user `b0df4a85f68d4cdf8e2d34ac4fbd3874`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:35:24.199194 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9c340ece-3ab1-4aa8-85ac-b51abaa9871f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:24.275539 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 899/1807] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:35:24 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:35:24.288857 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986cc4b4-cd35-48b5-b5cc-bfff63fa1d28 service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:24.291259 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-986cc4b4-cd35-48b5-b5cc-bfff63fa1d28 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:24.303836 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986cc4b4-cd35-48b5-b5cc-bfff63fa1d28 service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:24.307445 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec3950-0d59-4cd9-9c78-b07da1acedb6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:24.308144 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec3950-0d59-4cd9-9c78-b07da1acedb6 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:24.308433 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec3950-0d59-4cd9-9c78-b07da1acedb6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:24.308723 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdec3950-0d59-4cd9-9c78-b07da1acedb6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:24.391552 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdec3950-0d59-4cd9-9c78-b07da1acedb6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:24.393057 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdec3950-0d59-4cd9-9c78-b07da1acedb6 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:24.404076 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 909/1808] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:35:24 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:35:25.828703 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c340ece-3ab1-4aa8-85ac-b51abaa9871f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:25.838048 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c340ece-3ab1-4aa8-85ac-b51abaa9871f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:25.841627 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e519caf1-1d49-44d6-a3ce-80203783fbde service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:25.842227 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e519caf1-1d49-44d6-a3ce-80203783fbde service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:25.842501 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e519caf1-1d49-44d6-a3ce-80203783fbde service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:25.842796 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e519caf1-1d49-44d6-a3ce-80203783fbde service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:25.851530 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e519caf1-1d49-44d6-a3ce-80203783fbde service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:25.852972 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e519caf1-1d49-44d6-a3ce-80203783fbde service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:25.857800 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:35:25.863342 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 900/1809] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:35:25 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:35:27.833143 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec3950-0d59-4cd9-9c78-b07da1acedb6 service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:27.834582 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-cdec3950-0d59-4cd9-9c78-b07da1acedb6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:27.845888 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:35:27.902326 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdec3950-0d59-4cd9-9c78-b07da1acedb6 service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:27.909481 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed9b564-d52f-4bf9-8c53-5f7e86f64ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:27.910127 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9b564-d52f-4bf9-8c53-5f7e86f64ea7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:27.910410 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9b564-d52f-4bf9-8c53-5f7e86f64ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:27.910687 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed9b564-d52f-4bf9-8c53-5f7e86f64ea7 admin admin] PATH_INFO: `/v3/users/feb3d7c64d5247a58c9e1e606c62b476` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:27.911818 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed9b564-d52f-4bf9-8c53-5f7e86f64ea7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=feb3d7c64d5247a58c9e1e606c62b476)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:27.913773 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed9b564-d52f-4bf9-8c53-5f7e86f64ea7 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:27.964279 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-eed9b564-d52f-4bf9-8c53-5f7e86f64ea7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'feb3d7c64d5247a58c9e1e606c62b476'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:27.964932 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-eed9b564-d52f-4bf9-8c53-5f7e86f64ea7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'feb3d7c64d5247a58c9e1e606c62b476'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:27.965631 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-eed9b564-d52f-4bf9-8c53-5f7e86f64ea7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'feb3d7c64d5247a58c9e1e606c62b476'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:27.978665 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-eed9b564-d52f-4bf9-8c53-5f7e86f64ea7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'feb3d7c64d5247a58c9e1e606c62b476'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:27.986772 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-eed9b564-d52f-4bf9-8c53-5f7e86f64ea7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'feb3d7c64d5247a58c9e1e606c62b476'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:27.999308 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 910/1810] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 03:35:27 2022] DELETE /identity/v3/users/feb3d7c64d5247a58c9e1e606c62b476 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:35:28.003790 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49867630-ae5f-4c48-96b3-e03284c6a09e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:28.004281 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49867630-ae5f-4c48-96b3-e03284c6a09e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:28.004371 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49867630-ae5f-4c48-96b3-e03284c6a09e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:28.025302 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-49867630-ae5f-4c48-96b3-e03284c6a09e None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:35:28.026523 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-49867630-ae5f-4c48-96b3-e03284c6a09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:28.087087 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 901/1811] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:35:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:35:28.096745 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed9b564-d52f-4bf9-8c53-5f7e86f64ea7 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:28.098160 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-eed9b564-d52f-4bf9-8c53-5f7e86f64ea7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:28.155602 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed9b564-d52f-4bf9-8c53-5f7e86f64ea7 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:28.159429 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33247e96-7ca9-4edb-8cf0-bfe737cec06a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:28.160064 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33247e96-7ca9-4edb-8cf0-bfe737cec06a service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:28.160351 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33247e96-7ca9-4edb-8cf0-bfe737cec06a service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:28.160626 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33247e96-7ca9-4edb-8cf0-bfe737cec06a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:28.169761 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33247e96-7ca9-4edb-8cf0-bfe737cec06a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:28.171688 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33247e96-7ca9-4edb-8cf0-bfe737cec06a service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:28.181195 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 911/1812] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:35:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:35:28.461910 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49867630-ae5f-4c48-96b3-e03284c6a09e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:28.463353 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-49867630-ae5f-4c48-96b3-e03284c6a09e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:28.477273 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49867630-ae5f-4c48-96b3-e03284c6a09e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:28.482378 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72888685-eeaa-4790-b876-019a813757d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:28.483237 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72888685-eeaa-4790-b876-019a813757d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:28.483349 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72888685-eeaa-4790-b876-019a813757d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:28.484173 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72888685-eeaa-4790-b876-019a813757d6 admin admin] PATH_INFO: `/v3/projects/0b3a0a3d2512405b869c4f7f70927064` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:28.485366 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72888685-eeaa-4790-b876-019a813757d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b3a0a3d2512405b869c4f7f70927064)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:28.488059 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72888685-eeaa-4790-b876-019a813757d6 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:28.582437 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-72888685-eeaa-4790-b876-019a813757d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b3a0a3d2512405b869c4f7f70927064', 'request_id': 'req-72888685-eeaa-4790-b876-019a813757d6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:28.590858 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-72888685-eeaa-4790-b876-019a813757d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b3a0a3d2512405b869c4f7f70927064', 'request_id': 'req-72888685-eeaa-4790-b876-019a813757d6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:28.599596 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-72888685-eeaa-4790-b876-019a813757d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b3a0a3d2512405b869c4f7f70927064', 'request_id': 'req-72888685-eeaa-4790-b876-019a813757d6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:28.612287 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-72888685-eeaa-4790-b876-019a813757d6 admin admin] The token cache is being invalidate because project 0b3a0a3d2512405b869c4f7f70927064 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 03:35:28.612567 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-72888685-eeaa-4790-b876-019a813757d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:28.613274 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-72888685-eeaa-4790-b876-019a813757d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:28.614875 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 902/1813] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 03:35:28 2022] DELETE /identity/v3/projects/0b3a0a3d2512405b869c4f7f70927064 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:35:28.618934 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33247e96-7ca9-4edb-8cf0-bfe737cec06a service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:28.629806 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33247e96-7ca9-4edb-8cf0-bfe737cec06a service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:28.661808 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:35:28.694709 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad85bb7-20fe-48d9-a2e7-fe5d4d0cafed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:28.695647 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad85bb7-20fe-48d9-a2e7-fe5d4d0cafed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:28.696015 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad85bb7-20fe-48d9-a2e7-fe5d4d0cafed admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:28.696355 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad85bb7-20fe-48d9-a2e7-fe5d4d0cafed admin admin] PATH_INFO: `/v3/users/f6e5b241d57348bea96ed8bb7cf9d6c0` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:28.697663 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad85bb7-20fe-48d9-a2e7-fe5d4d0cafed admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6e5b241d57348bea96ed8bb7cf9d6c0)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:28.699675 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad85bb7-20fe-48d9-a2e7-fe5d4d0cafed admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:28.754714 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3ad85bb7-20fe-48d9-a2e7-fe5d4d0cafed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6e5b241d57348bea96ed8bb7cf9d6c0'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:28.755378 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3ad85bb7-20fe-48d9-a2e7-fe5d4d0cafed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6e5b241d57348bea96ed8bb7cf9d6c0'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:28.756006 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3ad85bb7-20fe-48d9-a2e7-fe5d4d0cafed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6e5b241d57348bea96ed8bb7cf9d6c0'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:28.768058 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3ad85bb7-20fe-48d9-a2e7-fe5d4d0cafed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6e5b241d57348bea96ed8bb7cf9d6c0'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:28.775125 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3ad85bb7-20fe-48d9-a2e7-fe5d4d0cafed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6e5b241d57348bea96ed8bb7cf9d6c0'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:28.787076 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 912/1814] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 03:35:28 2022] DELETE /identity/v3/users/f6e5b241d57348bea96ed8bb7cf9d6c0 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:35:29.058920 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72888685-eeaa-4790-b876-019a813757d6 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:29.061354 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-72888685-eeaa-4790-b876-019a813757d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:29.082205 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72888685-eeaa-4790-b876-019a813757d6 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:29.143540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5928903b-0ab8-4957-b979-cb37f870059f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:29.144239 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5928903b-0ab8-4957-b979-cb37f870059f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:29.144527 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5928903b-0ab8-4957-b979-cb37f870059f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:29.144799 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5928903b-0ab8-4957-b979-cb37f870059f admin admin] PATH_INFO: `/v3/projects/617a422546074a37b8f5609195512bef` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:29.145933 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5928903b-0ab8-4957-b979-cb37f870059f admin admin] RBAC: Authorizing `identity:delete_project(project_id=617a422546074a37b8f5609195512bef)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:29.147844 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5928903b-0ab8-4957-b979-cb37f870059f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:29.179373 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:35:29.199236 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5928903b-0ab8-4957-b979-cb37f870059f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '617a422546074a37b8f5609195512bef', 'request_id': 'req-5928903b-0ab8-4957-b979-cb37f870059f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:29.206709 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5928903b-0ab8-4957-b979-cb37f870059f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '617a422546074a37b8f5609195512bef', 'request_id': 'req-5928903b-0ab8-4957-b979-cb37f870059f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:29.214542 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5928903b-0ab8-4957-b979-cb37f870059f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '617a422546074a37b8f5609195512bef', 'request_id': 'req-5928903b-0ab8-4957-b979-cb37f870059f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:29.225115 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5928903b-0ab8-4957-b979-cb37f870059f admin admin] The token cache is being invalidate because project 617a422546074a37b8f5609195512bef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 03:35:29.225403 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5928903b-0ab8-4957-b979-cb37f870059f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:29.226026 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5928903b-0ab8-4957-b979-cb37f870059f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:29.227511 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 903/1815] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 03:35:29 2022] DELETE /identity/v3/projects/617a422546074a37b8f5609195512bef => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:35:29.231778 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad85bb7-20fe-48d9-a2e7-fe5d4d0cafed admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:29.233420 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-3ad85bb7-20fe-48d9-a2e7-fe5d4d0cafed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:29.247511 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad85bb7-20fe-48d9-a2e7-fe5d4d0cafed admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:29.308737 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5eb1d5-9f9a-4aca-8a81-cab589720d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:29.309394 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5eb1d5-9f9a-4aca-8a81-cab589720d14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:29.309679 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5eb1d5-9f9a-4aca-8a81-cab589720d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:29.310106 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5eb1d5-9f9a-4aca-8a81-cab589720d14 admin admin] PATH_INFO: `/v3/users/af798437fbbd4793985f43fc4199ad51` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:29.311084 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5eb1d5-9f9a-4aca-8a81-cab589720d14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af798437fbbd4793985f43fc4199ad51)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:29.313682 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5eb1d5-9f9a-4aca-8a81-cab589720d14 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:29.371678 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-8a5eb1d5-9f9a-4aca-8a81-cab589720d14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af798437fbbd4793985f43fc4199ad51'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:29.372357 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-8a5eb1d5-9f9a-4aca-8a81-cab589720d14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af798437fbbd4793985f43fc4199ad51'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:29.373246 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-8a5eb1d5-9f9a-4aca-8a81-cab589720d14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af798437fbbd4793985f43fc4199ad51'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:29.378849 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:35:29.388786 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-8a5eb1d5-9f9a-4aca-8a81-cab589720d14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af798437fbbd4793985f43fc4199ad51'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:29.396787 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-8a5eb1d5-9f9a-4aca-8a81-cab589720d14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af798437fbbd4793985f43fc4199ad51'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:29.410528 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 913/1816] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 03:35:29 2022] DELETE /identity/v3/users/af798437fbbd4793985f43fc4199ad51 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 03:35:29.765905 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5928903b-0ab8-4957-b979-cb37f870059f admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:29.767391 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5928903b-0ab8-4957-b979-cb37f870059f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:29.782822 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5928903b-0ab8-4957-b979-cb37f870059f admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:29.860605 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb92bdc-643c-48cf-a9d3-b9acd257b340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:29.861422 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb92bdc-643c-48cf-a9d3-b9acd257b340 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:29.861852 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb92bdc-643c-48cf-a9d3-b9acd257b340 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:29.862154 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb92bdc-643c-48cf-a9d3-b9acd257b340 admin admin] PATH_INFO: `/v3/projects/22fa95e6a9614b6ba1d969f5b008bd16` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:29.864133 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb92bdc-643c-48cf-a9d3-b9acd257b340 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22fa95e6a9614b6ba1d969f5b008bd16)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:29.866142 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb92bdc-643c-48cf-a9d3-b9acd257b340 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:29.921464 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-beb92bdc-643c-48cf-a9d3-b9acd257b340 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22fa95e6a9614b6ba1d969f5b008bd16', 'request_id': 'req-beb92bdc-643c-48cf-a9d3-b9acd257b340'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:29.929382 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-beb92bdc-643c-48cf-a9d3-b9acd257b340 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22fa95e6a9614b6ba1d969f5b008bd16', 'request_id': 'req-beb92bdc-643c-48cf-a9d3-b9acd257b340'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:29.939283 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-beb92bdc-643c-48cf-a9d3-b9acd257b340 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22fa95e6a9614b6ba1d969f5b008bd16', 'request_id': 'req-beb92bdc-643c-48cf-a9d3-b9acd257b340'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:29.952299 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-beb92bdc-643c-48cf-a9d3-b9acd257b340 admin admin] The token cache is being invalidate because project 22fa95e6a9614b6ba1d969f5b008bd16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 03:35:29.952530 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-beb92bdc-643c-48cf-a9d3-b9acd257b340 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:29.953192 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-beb92bdc-643c-48cf-a9d3-b9acd257b340 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:29.954723 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 904/1817] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 03:35:29 2022] DELETE /identity/v3/projects/22fa95e6a9614b6ba1d969f5b008bd16 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:35:33.330768 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c33f3-7e05-48f1-b6bc-f4fd15144626 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:33.331284 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c33f3-7e05-48f1-b6bc-f4fd15144626 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:33.331284 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304c33f3-7e05-48f1-b6bc-f4fd15144626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:33.351814 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-304c33f3-7e05-48f1-b6bc-f4fd15144626 None None] MFA Rules not processed for user `7984dc1d452a4c998b44e6d26a299df8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:35:33.353107 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-304c33f3-7e05-48f1-b6bc-f4fd15144626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:33.411659 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 914/1818] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:35:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:35:38.854064 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de8ef14-f03b-4709-914f-549a7052b6f6 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:38.854883 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de8ef14-f03b-4709-914f-549a7052b6f6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:38.855585 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de8ef14-f03b-4709-914f-549a7052b6f6 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:38.857634 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 905/1819] 10.222.0.26 () {54 vars in 921 bytes} [Wed Nov 2 03:35:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:35:38.883041 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712555a9-3990-48b5-9c4c-9a9fee4cfa38 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:38.883497 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712555a9-3990-48b5-9c4c-9a9fee4cfa38 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:38.883769 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712555a9-3990-48b5-9c4c-9a9fee4cfa38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:38.918481 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-712555a9-3990-48b5-9c4c-9a9fee4cfa38 None None] MFA Rules not processed for user `9b4b9bd741574ddabadd4b7651ebc885`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:35:38.919702 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-712555a9-3990-48b5-9c4c-9a9fee4cfa38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:39.021681 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 915/1820] 10.222.0.26 () {58 vars in 1018 bytes} [Wed Nov 2 03:35:38 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:35:39.041488 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de8ef14-f03b-4709-914f-549a7052b6f6 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:39.043326 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4de8ef14-f03b-4709-914f-549a7052b6f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:39.063401 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de8ef14-f03b-4709-914f-549a7052b6f6 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:39.069710 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51af2893-9485-40c2-a12b-2bdaec2352e7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4b9bd741574ddabadd4b7651ebc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:39.070437 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af2893-9485-40c2-a12b-2bdaec2352e7 service placement] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:39.070880 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af2893-9485-40c2-a12b-2bdaec2352e7 service placement] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:39.071248 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af2893-9485-40c2-a12b-2bdaec2352e7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:39.080832 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51af2893-9485-40c2-a12b-2bdaec2352e7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:39.082596 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51af2893-9485-40c2-a12b-2bdaec2352e7 service placement] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:39.092701 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 906/1821] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:35:39 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:35:39.767093 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712555a9-3990-48b5-9c4c-9a9fee4cfa38 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:39.768591 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-712555a9-3990-48b5-9c4c-9a9fee4cfa38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:39.836740 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712555a9-3990-48b5-9c4c-9a9fee4cfa38 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:39.841326 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041ea9bc-dae6-4df0-8613-b34197023627 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:39.841967 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041ea9bc-dae6-4df0-8613-b34197023627 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:39.842248 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041ea9bc-dae6-4df0-8613-b34197023627 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:39.843112 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041ea9bc-dae6-4df0-8613-b34197023627 admin admin] PATH_INFO: `/v3/users/737aa5b75c714fe7ad3a1229e9b19700` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:39.844275 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041ea9bc-dae6-4df0-8613-b34197023627 admin admin] RBAC: Authorizing `identity:delete_user(user_id=737aa5b75c714fe7ad3a1229e9b19700)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:39.846203 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041ea9bc-dae6-4df0-8613-b34197023627 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:39.859755 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:35:39.895730 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-041ea9bc-dae6-4df0-8613-b34197023627 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '737aa5b75c714fe7ad3a1229e9b19700'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:39.896602 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-041ea9bc-dae6-4df0-8613-b34197023627 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '737aa5b75c714fe7ad3a1229e9b19700'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:39.897230 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-041ea9bc-dae6-4df0-8613-b34197023627 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '737aa5b75c714fe7ad3a1229e9b19700'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:39.909822 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-041ea9bc-dae6-4df0-8613-b34197023627 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '737aa5b75c714fe7ad3a1229e9b19700'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:39.919751 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-041ea9bc-dae6-4df0-8613-b34197023627 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '737aa5b75c714fe7ad3a1229e9b19700'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:39.932975 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 916/1822] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 03:35:39 2022] DELETE /identity/v3/users/737aa5b75c714fe7ad3a1229e9b19700 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:35:40.103215 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51af2893-9485-40c2-a12b-2bdaec2352e7 service placement] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:40.114232 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51af2893-9485-40c2-a12b-2bdaec2352e7 service placement] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:40.144296 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:35:40.174323 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49353b3-2676-4161-acbe-88f88f6b482d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:40.174967 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49353b3-2676-4161-acbe-88f88f6b482d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:40.175418 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49353b3-2676-4161-acbe-88f88f6b482d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:40.175689 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49353b3-2676-4161-acbe-88f88f6b482d admin admin] PATH_INFO: `/v3/projects/513566a027164bab9f21f49f96bb878d` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:40.177015 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49353b3-2676-4161-acbe-88f88f6b482d admin admin] RBAC: Authorizing `identity:delete_project(project_id=513566a027164bab9f21f49f96bb878d)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:40.179205 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49353b3-2676-4161-acbe-88f88f6b482d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:40.227995 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f49353b3-2676-4161-acbe-88f88f6b482d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '513566a027164bab9f21f49f96bb878d', 'request_id': 'req-f49353b3-2676-4161-acbe-88f88f6b482d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:40.234332 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f49353b3-2676-4161-acbe-88f88f6b482d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '513566a027164bab9f21f49f96bb878d', 'request_id': 'req-f49353b3-2676-4161-acbe-88f88f6b482d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:40.242188 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f49353b3-2676-4161-acbe-88f88f6b482d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '513566a027164bab9f21f49f96bb878d', 'request_id': 'req-f49353b3-2676-4161-acbe-88f88f6b482d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:40.253195 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f49353b3-2676-4161-acbe-88f88f6b482d admin admin] The token cache is being invalidate because project 513566a027164bab9f21f49f96bb878d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 03:35:40.253469 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f49353b3-2676-4161-acbe-88f88f6b482d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:40.254176 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f49353b3-2676-4161-acbe-88f88f6b482d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:40.255710 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 907/1823] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 03:35:40 2022] DELETE /identity/v3/projects/513566a027164bab9f21f49f96bb878d => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:35:40.259733 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041ea9bc-dae6-4df0-8613-b34197023627 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:40.261399 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-041ea9bc-dae6-4df0-8613-b34197023627 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:40.274286 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041ea9bc-dae6-4df0-8613-b34197023627 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:40.339834 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3467b243-394c-47ae-9856-7851eb8ffe6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:40.340657 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3467b243-394c-47ae-9856-7851eb8ffe6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:40.340939 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3467b243-394c-47ae-9856-7851eb8ffe6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:40.341215 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3467b243-394c-47ae-9856-7851eb8ffe6b admin admin] PATH_INFO: `/v3/users/b0df4a85f68d4cdf8e2d34ac4fbd3874` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:40.342337 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3467b243-394c-47ae-9856-7851eb8ffe6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0df4a85f68d4cdf8e2d34ac4fbd3874)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:40.344832 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3467b243-394c-47ae-9856-7851eb8ffe6b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:40.389478 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3467b243-394c-47ae-9856-7851eb8ffe6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0df4a85f68d4cdf8e2d34ac4fbd3874'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:40.390120 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3467b243-394c-47ae-9856-7851eb8ffe6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0df4a85f68d4cdf8e2d34ac4fbd3874'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:40.390723 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3467b243-394c-47ae-9856-7851eb8ffe6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0df4a85f68d4cdf8e2d34ac4fbd3874'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:40.402619 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3467b243-394c-47ae-9856-7851eb8ffe6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0df4a85f68d4cdf8e2d34ac4fbd3874'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:40.409975 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3467b243-394c-47ae-9856-7851eb8ffe6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0df4a85f68d4cdf8e2d34ac4fbd3874'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:40.422374 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 917/1824] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 03:35:40 2022] DELETE /identity/v3/users/b0df4a85f68d4cdf8e2d34ac4fbd3874 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:35:40.598613 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49353b3-2676-4161-acbe-88f88f6b482d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:40.600089 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f49353b3-2676-4161-acbe-88f88f6b482d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:40.615547 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49353b3-2676-4161-acbe-88f88f6b482d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:40.677303 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f3e6ac-104a-45c6-b0b7-36e383d5ca23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:40.678361 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f3e6ac-104a-45c6-b0b7-36e383d5ca23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:40.678655 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f3e6ac-104a-45c6-b0b7-36e383d5ca23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:40.678935 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f3e6ac-104a-45c6-b0b7-36e383d5ca23 admin admin] PATH_INFO: `/v3/projects/7fb66b88418141aca86a24340356dec6` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:40.680114 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f3e6ac-104a-45c6-b0b7-36e383d5ca23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fb66b88418141aca86a24340356dec6)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:40.682020 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f3e6ac-104a-45c6-b0b7-36e383d5ca23 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:40.728865 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-50f3e6ac-104a-45c6-b0b7-36e383d5ca23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fb66b88418141aca86a24340356dec6', 'request_id': 'req-50f3e6ac-104a-45c6-b0b7-36e383d5ca23'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:40.734990 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-50f3e6ac-104a-45c6-b0b7-36e383d5ca23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fb66b88418141aca86a24340356dec6', 'request_id': 'req-50f3e6ac-104a-45c6-b0b7-36e383d5ca23'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:40.742208 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-50f3e6ac-104a-45c6-b0b7-36e383d5ca23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fb66b88418141aca86a24340356dec6', 'request_id': 'req-50f3e6ac-104a-45c6-b0b7-36e383d5ca23'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:40.744180 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:35:40.754310 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-50f3e6ac-104a-45c6-b0b7-36e383d5ca23 admin admin] The token cache is being invalidate because project 7fb66b88418141aca86a24340356dec6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 03:35:40.754596 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-50f3e6ac-104a-45c6-b0b7-36e383d5ca23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:40.755241 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-50f3e6ac-104a-45c6-b0b7-36e383d5ca23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:35:40.756753 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 908/1825] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 03:35:40 2022] DELETE /identity/v3/projects/7fb66b88418141aca86a24340356dec6 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:35:40.761706 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13bf6a2-6ab0-420d-a859-e253ff815d71 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:40.762090 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13bf6a2-6ab0-420d-a859-e253ff815d71 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:40.762372 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13bf6a2-6ab0-420d-a859-e253ff815d71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:40.782783 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-e13bf6a2-6ab0-420d-a859-e253ff815d71 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:35:40.784008 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e13bf6a2-6ab0-420d-a859-e253ff815d71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:40.788110 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:35:40.845815 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 918/1826] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:35:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:35:40.853477 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd0fbc-ce5f-4f7a-a1a2-ebedf4a34412 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:40.853767 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd0fbc-ce5f-4f7a-a1a2-ebedf4a34412 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:40.854045 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdd0fbc-ce5f-4f7a-a1a2-ebedf4a34412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:40.875219 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-5bdd0fbc-ce5f-4f7a-a1a2-ebedf4a34412 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:35:40.876480 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5bdd0fbc-ce5f-4f7a-a1a2-ebedf4a34412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:40.884506 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 909/1827] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:35:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:35:40.889209 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13bf6a2-6ab0-420d-a859-e253ff815d71 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:40.900810 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13bf6a2-6ab0-420d-a859-e253ff815d71 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:40.907048 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209d64be-90df-4080-b525-4d173aad8b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:40.907672 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d64be-90df-4080-b525-4d173aad8b54 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:40.907977 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d64be-90df-4080-b525-4d173aad8b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:40.908247 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209d64be-90df-4080-b525-4d173aad8b54 admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:40.908901 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209d64be-90df-4080-b525-4d173aad8b54 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:35:40.909203 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209d64be-90df-4080-b525-4d173aad8b54 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:40.910802 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209d64be-90df-4080-b525-4d173aad8b54 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:40.922857 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 919/1828] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 03:35:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:35:40.927382 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdd0fbc-ce5f-4f7a-a1a2-ebedf4a34412 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:40.937522 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdd0fbc-ce5f-4f7a-a1a2-ebedf4a34412 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:40.942300 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4031dec8-f6b6-47b7-989b-55a99820e38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:40.942911 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4031dec8-f6b6-47b7-989b-55a99820e38b admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:40.943219 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4031dec8-f6b6-47b7-989b-55a99820e38b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:40.943492 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4031dec8-f6b6-47b7-989b-55a99820e38b admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:40.944185 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4031dec8-f6b6-47b7-989b-55a99820e38b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:40.946118 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4031dec8-f6b6-47b7-989b-55a99820e38b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:40.980990 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 910/1829] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:35:40 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 03:35:40.986012 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209d64be-90df-4080-b525-4d173aad8b54 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:40.997355 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209d64be-90df-4080-b525-4d173aad8b54 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:41.064097 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe65126-623a-4f87-a2ca-bc71e2435397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:41.064715 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe65126-623a-4f87-a2ca-bc71e2435397 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:41.064997 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe65126-623a-4f87-a2ca-bc71e2435397 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:41.065266 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe65126-623a-4f87-a2ca-bc71e2435397 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:41.065910 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe65126-623a-4f87-a2ca-bc71e2435397 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:41.067865 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe65126-623a-4f87-a2ca-bc71e2435397 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:41.089449 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 920/1830] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:35:40 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:35:41.093549 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4031dec8-f6b6-47b7-989b-55a99820e38b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:41.104196 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4031dec8-f6b6-47b7-989b-55a99820e38b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:41.108848 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f8d57e-2787-42a1-86d5-dec1db095567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:41.109460 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8d57e-2787-42a1-86d5-dec1db095567 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:41.109741 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8d57e-2787-42a1-86d5-dec1db095567 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:41.110013 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8d57e-2787-42a1-86d5-dec1db095567 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:41.110655 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f8d57e-2787-42a1-86d5-dec1db095567 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:41.112305 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f8d57e-2787-42a1-86d5-dec1db095567 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:41.122488 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 911/1831] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:35:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:35:41.127212 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe65126-623a-4f87-a2ca-bc71e2435397 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:41.137365 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe65126-623a-4f87-a2ca-bc71e2435397 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:41.142755 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108b7b0f-6897-446a-84e4-4c178f7bf9ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:41.143366 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b7b0f-6897-446a-84e4-4c178f7bf9ed admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:41.143649 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b7b0f-6897-446a-84e4-4c178f7bf9ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:41.143932 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108b7b0f-6897-446a-84e4-4c178f7bf9ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:41.144609 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108b7b0f-6897-446a-84e4-4c178f7bf9ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:41.146209 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108b7b0f-6897-446a-84e4-4c178f7bf9ed admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:41.159431 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 921/1832] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:35:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:35:41.163476 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f8d57e-2787-42a1-86d5-dec1db095567 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:41.173858 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f8d57e-2787-42a1-86d5-dec1db095567 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:41.178564 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341a070a-7e1d-43a8-9df9-428eee3ebda7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:41.179245 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a070a-7e1d-43a8-9df9-428eee3ebda7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:41.179526 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a070a-7e1d-43a8-9df9-428eee3ebda7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:41.179796 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341a070a-7e1d-43a8-9df9-428eee3ebda7 admin admin] PATH_INFO: `/v3/projects/0429960aa0454ea38fea17c0302a9a8a/users/c44fc29f3a014d8c9ca46903a4ad4211/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:41.194987 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341a070a-7e1d-43a8-9df9-428eee3ebda7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0429960aa0454ea38fea17c0302a9a8a, user_id=c44fc29f3a014d8c9ca46903a4ad4211, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:41.197676 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341a070a-7e1d-43a8-9df9-428eee3ebda7 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:41.211256 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 912/1833] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:35:41 2022] PUT /identity/v3/projects/0429960aa0454ea38fea17c0302a9a8a/users/c44fc29f3a014d8c9ca46903a4ad4211/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:35:41.222578 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c95d25-e06e-48cb-9693-be121eecc614 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:41.222866 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c95d25-e06e-48cb-9693-be121eecc614 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:41.223143 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c95d25-e06e-48cb-9693-be121eecc614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:41.251072 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-a9c95d25-e06e-48cb-9693-be121eecc614 None None] MFA Rules not processed for user `c44fc29f3a014d8c9ca46903a4ad4211`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:35:41.252320 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a9c95d25-e06e-48cb-9693-be121eecc614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:41.316975 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 922/1834] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:35:41 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:35:41.321461 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341a070a-7e1d-43a8-9df9-428eee3ebda7 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:41.392080 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341a070a-7e1d-43a8-9df9-428eee3ebda7 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:41.396751 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce8c36c-de5e-4ae7-a2b2-d74f9b7e7057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:41.397363 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce8c36c-de5e-4ae7-a2b2-d74f9b7e7057 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:41.397641 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce8c36c-de5e-4ae7-a2b2-d74f9b7e7057 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:41.397910 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce8c36c-de5e-4ae7-a2b2-d74f9b7e7057 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:41.398547 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce8c36c-de5e-4ae7-a2b2-d74f9b7e7057 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:41.400483 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce8c36c-de5e-4ae7-a2b2-d74f9b7e7057 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:41.431013 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 913/1835] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:35:41 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 03:35:41.432280 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c95d25-e06e-48cb-9693-be121eecc614 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:41.439718 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:35:41.444744 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c95d25-e06e-48cb-9693-be121eecc614 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:41.514310 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f048f9a-f451-4081-8b51-eb2aa44640c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:41.515039 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f048f9a-f451-4081-8b51-eb2aa44640c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:41.515381 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f048f9a-f451-4081-8b51-eb2aa44640c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:41.515723 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f048f9a-f451-4081-8b51-eb2aa44640c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:41.516479 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f048f9a-f451-4081-8b51-eb2aa44640c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:41.518451 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f048f9a-f451-4081-8b51-eb2aa44640c5 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:41.547780 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 923/1836] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:35:41 2022] POST /identity/v3/users => generated 299 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 03:35:41.548790 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce8c36c-de5e-4ae7-a2b2-d74f9b7e7057 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:41.560523 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce8c36c-de5e-4ae7-a2b2-d74f9b7e7057 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:41.578065 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e1c33b-14a7-4b83-a8f0-2815845cdebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:41.578821 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e1c33b-14a7-4b83-a8f0-2815845cdebd admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:41.579108 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e1c33b-14a7-4b83-a8f0-2815845cdebd admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:41.579413 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e1c33b-14a7-4b83-a8f0-2815845cdebd admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:41.580133 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e1c33b-14a7-4b83-a8f0-2815845cdebd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:41.581735 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e1c33b-14a7-4b83-a8f0-2815845cdebd admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:41.593068 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 914/1837] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:35:41 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:35:41.597211 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f048f9a-f451-4081-8b51-eb2aa44640c5 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:41.607407 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f048f9a-f451-4081-8b51-eb2aa44640c5 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:41.612546 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d40ac3-80d1-4290-b08d-2d2c7cdbd50c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:41.613201 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d40ac3-80d1-4290-b08d-2d2c7cdbd50c admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:41.613484 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d40ac3-80d1-4290-b08d-2d2c7cdbd50c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:41.613765 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d40ac3-80d1-4290-b08d-2d2c7cdbd50c admin admin] PATH_INFO: `/v3/projects/e34b33f691d84080834e8375928b171b/users/e9d5f64e2ae34034802033194e84f192/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:41.629568 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d40ac3-80d1-4290-b08d-2d2c7cdbd50c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e34b33f691d84080834e8375928b171b, user_id=e9d5f64e2ae34034802033194e84f192, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:41.632259 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d40ac3-80d1-4290-b08d-2d2c7cdbd50c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:41.644915 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 924/1838] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:35:41 2022] PUT /identity/v3/projects/e34b33f691d84080834e8375928b171b/users/e9d5f64e2ae34034802033194e84f192/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:35:41.648824 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e1c33b-14a7-4b83-a8f0-2815845cdebd admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:41.659082 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e1c33b-14a7-4b83-a8f0-2815845cdebd admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:41.723299 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2b1abc-7633-4db6-ae6f-f686ce94f8fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:41.723926 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b1abc-7633-4db6-ae6f-f686ce94f8fb admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:41.724238 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b1abc-7633-4db6-ae6f-f686ce94f8fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:41.724625 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b1abc-7633-4db6-ae6f-f686ce94f8fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:41.725318 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2b1abc-7633-4db6-ae6f-f686ce94f8fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:41.726967 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2b1abc-7633-4db6-ae6f-f686ce94f8fb admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:41.729540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:35:41.741372 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 915/1839] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:35:41 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:35:41.743832 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d40ac3-80d1-4290-b08d-2d2c7cdbd50c admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:41.754681 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d40ac3-80d1-4290-b08d-2d2c7cdbd50c admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:41.759355 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01a0baf-bea9-413b-8d14-1f2cf8f4e374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:41.760083 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a0baf-bea9-413b-8d14-1f2cf8f4e374 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:41.760365 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a0baf-bea9-413b-8d14-1f2cf8f4e374 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:41.760656 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a0baf-bea9-413b-8d14-1f2cf8f4e374 admin admin] PATH_INFO: `/v3/projects/e34b33f691d84080834e8375928b171b/users/e9d5f64e2ae34034802033194e84f192/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:41.762587 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01a0baf-bea9-413b-8d14-1f2cf8f4e374 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e34b33f691d84080834e8375928b171b, user_id=e9d5f64e2ae34034802033194e84f192, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:41.765303 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01a0baf-bea9-413b-8d14-1f2cf8f4e374 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:41.784201 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 925/1840] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:35:41 2022] PUT /identity/v3/projects/e34b33f691d84080834e8375928b171b/users/e9d5f64e2ae34034802033194e84f192/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:35:41.786927 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8aca0a-e804-4269-972e-2cad28668a77 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:41.787216 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8aca0a-e804-4269-972e-2cad28668a77 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:41.787491 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8aca0a-e804-4269-972e-2cad28668a77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:41.816179 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-8f8aca0a-e804-4269-972e-2cad28668a77 None None] MFA Rules not processed for user `e9d5f64e2ae34034802033194e84f192`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:35:41.817408 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8f8aca0a-e804-4269-972e-2cad28668a77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:41.884415 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 916/1841] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:35:41 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:35:41.895451 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01a0baf-bea9-413b-8d14-1f2cf8f4e374 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:35:41.897170 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-c01a0baf-bea9-413b-8d14-1f2cf8f4e374 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:35:41.954201 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01a0baf-bea9-413b-8d14-1f2cf8f4e374 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:35:41.957955 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af828364-270d-4e51-8ab0-84fabfede800 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:35:41.958562 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af828364-270d-4e51-8ab0-84fabfede800 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:35:41.958843 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af828364-270d-4e51-8ab0-84fabfede800 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:35:41.959116 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af828364-270d-4e51-8ab0-84fabfede800 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:35:42.014804 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af828364-270d-4e51-8ab0-84fabfede800 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:35:42.016264 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af828364-270d-4e51-8ab0-84fabfede800 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:35:42.024956 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 926/1842] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:35:41 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:35:51.881225 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=35 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:35:52.020916 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=90,keystone:INSERT=5 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:36:37.792688 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644089ed-25de-4e72-9fb2-1cc775139142 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:36:37.793196 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644089ed-25de-4e72-9fb2-1cc775139142 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:36:37.793377 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644089ed-25de-4e72-9fb2-1cc775139142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:36:37.813336 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-644089ed-25de-4e72-9fb2-1cc775139142 None None] MFA Rules not processed for user `da7ee607b66e43f3823d9d2faf373974`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:36:37.814532 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-644089ed-25de-4e72-9fb2-1cc775139142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:36:37.869139 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 917/1843] 10.222.0.26 () {58 vars in 1009 bytes} [Wed Nov 2 03:36:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:36:37.909121 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af828364-270d-4e51-8ab0-84fabfede800 service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:36:37.910549 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-af828364-270d-4e51-8ab0-84fabfede800 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:36:37.979658 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af828364-270d-4e51-8ab0-84fabfede800 service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:36:37.984743 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d700449-0a99-4fd0-af52-342d09a4c252 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5193c40a917b4105ab8b27aead1685f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:36:37.985357 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d700449-0a99-4fd0-af52-342d09a4c252 service swift] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:36:37.985638 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d700449-0a99-4fd0-af52-342d09a4c252 service swift] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:36:37.985913 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d700449-0a99-4fd0-af52-342d09a4c252 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:36:37.995477 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d700449-0a99-4fd0-af52-342d09a4c252 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:36:37.997100 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d700449-0a99-4fd0-af52-342d09a4c252 service swift] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:36:38.005488 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 927/1844] 10.222.0.26 () {60 vars in 1384 bytes} [Wed Nov 2 03:36:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Nov 02 03:36:43.077309 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644089ed-25de-4e72-9fb2-1cc775139142 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:36:43.078749 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-644089ed-25de-4e72-9fb2-1cc775139142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:36:43.203824 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644089ed-25de-4e72-9fb2-1cc775139142 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:36:43.210480 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cc6254-c7e1-45e8-ab13-a39dc000409a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4b9bd741574ddabadd4b7651ebc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:36:43.212538 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cc6254-c7e1-45e8-ab13-a39dc000409a service placement] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:36:43.212990 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cc6254-c7e1-45e8-ab13-a39dc000409a service placement] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:36:43.213431 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cc6254-c7e1-45e8-ab13-a39dc000409a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:36:43.215579 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-71cc6254-c7e1-45e8-ab13-a39dc000409a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:36:43.239553 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cc6254-c7e1-45e8-ab13-a39dc000409a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:36:43.244472 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cc6254-c7e1-45e8-ab13-a39dc000409a service placement] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:36:43.275976 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 918/1845] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:36:43 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:36:48.002110 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=32 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:36:52.180286 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973cd32d-3f90-47ae-8274-7a2756f31c52 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:36:52.180923 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973cd32d-3f90-47ae-8274-7a2756f31c52 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:36:52.181262 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973cd32d-3f90-47ae-8274-7a2756f31c52 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:36:52.183248 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 928/1846] 10.222.0.26 () {54 vars in 921 bytes} [Wed Nov 2 03:36:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:36:52.190002 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e623a5c-9a94-4690-882c-ec3a9470eab3 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:36:52.190553 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e623a5c-9a94-4690-882c-ec3a9470eab3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:36:52.190648 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e623a5c-9a94-4690-882c-ec3a9470eab3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:36:52.215328 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-8e623a5c-9a94-4690-882c-ec3a9470eab3 None None] MFA Rules not processed for user `44961b06151c4da382fcd54dd1f1e146`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:36:52.216794 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8e623a5c-9a94-4690-882c-ec3a9470eab3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:36:52.289010 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 919/1847] 10.222.0.26 () {58 vars in 1018 bytes} [Wed Nov 2 03:36:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:36:52.302514 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973cd32d-3f90-47ae-8274-7a2756f31c52 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:36:52.304197 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-973cd32d-3f90-47ae-8274-7a2756f31c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:36:52.364241 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973cd32d-3f90-47ae-8274-7a2756f31c52 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:36:52.368473 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9823376d-3cb3-466f-9103-40fd507fb137 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50aab624ee9c4651a7cd42124bcc465f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:36:52.369219 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9823376d-3cb3-466f-9103-40fd507fb137 service glance] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:36:52.369496 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9823376d-3cb3-466f-9103-40fd507fb137 service glance] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:36:52.369769 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9823376d-3cb3-466f-9103-40fd507fb137 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:36:52.381399 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9823376d-3cb3-466f-9103-40fd507fb137 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:36:52.382813 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9823376d-3cb3-466f-9103-40fd507fb137 service glance] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:36:52.394175 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 929/1848] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:36:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:36:52.463271 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de4554c-4aeb-4b85-bf01-2a624d21ed59 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:36:52.463555 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de4554c-4aeb-4b85-bf01-2a624d21ed59 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:36:52.463829 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de4554c-4aeb-4b85-bf01-2a624d21ed59 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:36:52.465203 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 920/1849] 10.222.0.26 () {54 vars in 921 bytes} [Wed Nov 2 03:36:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:36:52.469862 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d24afe0-0714-4a27-99db-c74010eec2ab None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:36:52.470178 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d24afe0-0714-4a27-99db-c74010eec2ab None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:36:52.470455 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d24afe0-0714-4a27-99db-c74010eec2ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:36:52.490886 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-0d24afe0-0714-4a27-99db-c74010eec2ab None None] MFA Rules not processed for user `f07e1b51b00f42b7b74c3689268635cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:36:52.492163 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-0d24afe0-0714-4a27-99db-c74010eec2ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:36:52.546216 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 930/1850] 10.222.0.26 () {58 vars in 1018 bytes} [Wed Nov 2 03:36:52 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:36:52.558520 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de4554c-4aeb-4b85-bf01-2a624d21ed59 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:36:52.560259 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5de4554c-4aeb-4b85-bf01-2a624d21ed59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:36:52.574065 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de4554c-4aeb-4b85-bf01-2a624d21ed59 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:36:52.577579 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ef8bbf-5774-43a8-9f75-ac3be913a584 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:36:52.578177 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef8bbf-5774-43a8-9f75-ac3be913a584 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:36:52.578461 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef8bbf-5774-43a8-9f75-ac3be913a584 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:36:52.578737 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ef8bbf-5774-43a8-9f75-ac3be913a584 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:36:52.585730 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ef8bbf-5774-43a8-9f75-ac3be913a584 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:36:52.587170 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ef8bbf-5774-43a8-9f75-ac3be913a584 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:36:52.594952 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 921/1851] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:36:52 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:36:52.677341 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3b7381-e775-4ef5-8b9b-5d15b2441cdd None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:36:52.677629 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3b7381-e775-4ef5-8b9b-5d15b2441cdd None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:36:52.677906 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3b7381-e775-4ef5-8b9b-5d15b2441cdd None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:36:52.679301 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 931/1852] 10.222.0.26 () {54 vars in 921 bytes} [Wed Nov 2 03:36:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 03:36:52.684618 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0053cfd9-fb9e-489f-a999-4aa3d3f88465 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:36:52.684905 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0053cfd9-fb9e-489f-a999-4aa3d3f88465 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:36:52.685234 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0053cfd9-fb9e-489f-a999-4aa3d3f88465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:36:52.705342 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-0053cfd9-fb9e-489f-a999-4aa3d3f88465 None None] MFA Rules not processed for user `44961b06151c4da382fcd54dd1f1e146`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:36:52.706569 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0053cfd9-fb9e-489f-a999-4aa3d3f88465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:36:52.715799 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 922/1853] 10.222.0.26 () {58 vars in 1018 bytes} [Wed Nov 2 03:36:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:36:52.726024 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3b7381-e775-4ef5-8b9b-5d15b2441cdd None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:36:52.737122 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3b7381-e775-4ef5-8b9b-5d15b2441cdd None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:36:52.741201 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4184056-9370-4845-afbd-1823124345e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:36:52.741796 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4184056-9370-4845-afbd-1823124345e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:36:52.742072 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4184056-9370-4845-afbd-1823124345e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:36:52.742344 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4184056-9370-4845-afbd-1823124345e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:36:52.752553 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4184056-9370-4845-afbd-1823124345e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:36:52.753976 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4184056-9370-4845-afbd-1823124345e5 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:36:52.766747 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 932/1854] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:36:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:37:02.700390 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:37:02.761166 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=55 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:38:22.189399 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0053cfd9-fb9e-489f-a999-4aa3d3f88465 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:38:22.199046 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0053cfd9-fb9e-489f-a999-4aa3d3f88465 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:38:22.202610 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c70d3f-060a-4aba-a884-4a3ddc7c70bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:38:22.203213 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c70d3f-060a-4aba-a884-4a3ddc7c70bd service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:38:22.203495 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c70d3f-060a-4aba-a884-4a3ddc7c70bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:38:22.203770 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c70d3f-060a-4aba-a884-4a3ddc7c70bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:38:22.205504 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-91c70d3f-060a-4aba-a884-4a3ddc7c70bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:38:22.218632 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c70d3f-060a-4aba-a884-4a3ddc7c70bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:38:22.220092 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c70d3f-060a-4aba-a884-4a3ddc7c70bd service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:38:22.231484 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 923/1855] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:38:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:38:32.226700 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=6 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:38:33.955817 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4184056-9370-4845-afbd-1823124345e5 service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:38:33.957376 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d4184056-9370-4845-afbd-1823124345e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:38:33.972560 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4184056-9370-4845-afbd-1823124345e5 service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:38:33.978162 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e2ceb1-0382-4c5e-91cd-4f3b0cbf03c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:38:33.978757 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e2ceb1-0382-4c5e-91cd-4f3b0cbf03c0 service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:38:33.979035 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e2ceb1-0382-4c5e-91cd-4f3b0cbf03c0 service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:38:33.979310 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e2ceb1-0382-4c5e-91cd-4f3b0cbf03c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:38:33.981047 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-55e2ceb1-0382-4c5e-91cd-4f3b0cbf03c0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:38:34.043395 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e2ceb1-0382-4c5e-91cd-4f3b0cbf03c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:38:34.045005 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e2ceb1-0382-4c5e-91cd-4f3b0cbf03c0 service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:38:34.054241 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 933/1856] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:38:33 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:38:40.853464 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c70d3f-060a-4aba-a884-4a3ddc7c70bd service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:38:40.854877 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-91c70d3f-060a-4aba-a884-4a3ddc7c70bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:38:40.869228 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c70d3f-060a-4aba-a884-4a3ddc7c70bd service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:38:40.874983 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e65b00-0ab5-455d-9308-6496ee78930a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:38:40.875579 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e65b00-0ab5-455d-9308-6496ee78930a service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:38:40.875862 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e65b00-0ab5-455d-9308-6496ee78930a service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:38:40.876168 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e65b00-0ab5-455d-9308-6496ee78930a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:38:40.877838 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a7e65b00-0ab5-455d-9308-6496ee78930a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:38:40.942029 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e65b00-0ab5-455d-9308-6496ee78930a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:38:40.943602 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e65b00-0ab5-455d-9308-6496ee78930a service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:38:40.952860 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 924/1857] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:38:40 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:38:44.049845 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=28 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:38:46.633127 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e2ceb1-0382-4c5e-91cd-4f3b0cbf03c0 service nova] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:38:46.642291 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e2ceb1-0382-4c5e-91cd-4f3b0cbf03c0 service nova] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:38:46.645802 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762f7d2f-2efc-4b90-b949-0fa02132c655 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:38:46.646397 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762f7d2f-2efc-4b90-b949-0fa02132c655 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:38:46.646675 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762f7d2f-2efc-4b90-b949-0fa02132c655 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:38:46.646943 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762f7d2f-2efc-4b90-b949-0fa02132c655 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:38:46.649353 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-762f7d2f-2efc-4b90-b949-0fa02132c655 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:38:46.707388 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762f7d2f-2efc-4b90-b949-0fa02132c655 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:38:46.708860 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762f7d2f-2efc-4b90-b949-0fa02132c655 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:38:46.718537 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 934/1858] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:38:46 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 03:38:48.303560 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e65b00-0ab5-455d-9308-6496ee78930a service nova] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:38:48.312843 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e65b00-0ab5-455d-9308-6496ee78930a service nova] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:38:48.316411 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d02e1ac-7eb5-4a67-ad0f-fbb237cfb6fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:38:48.317009 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d02e1ac-7eb5-4a67-ad0f-fbb237cfb6fd service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:38:48.317289 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d02e1ac-7eb5-4a67-ad0f-fbb237cfb6fd service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:38:48.317561 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d02e1ac-7eb5-4a67-ad0f-fbb237cfb6fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:38:48.319787 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5d02e1ac-7eb5-4a67-ad0f-fbb237cfb6fd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:38:48.384971 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d02e1ac-7eb5-4a67-ad0f-fbb237cfb6fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:38:48.386565 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d02e1ac-7eb5-4a67-ad0f-fbb237cfb6fd service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:38:48.396157 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 925/1859] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:38:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:38:56.714370 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=28 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:38:58.392525 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=56 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:39:18.961292 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762f7d2f-2efc-4b90-b949-0fa02132c655 service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:39:18.970987 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762f7d2f-2efc-4b90-b949-0fa02132c655 service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:39:18.974865 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2cf7a6-4ce5-4b28-a8de-c4414c06b791 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:39:18.975668 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2cf7a6-4ce5-4b28-a8de-c4414c06b791 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:39:18.975966 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2cf7a6-4ce5-4b28-a8de-c4414c06b791 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:39:18.976249 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2cf7a6-4ce5-4b28-a8de-c4414c06b791 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:39:18.978489 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ce2cf7a6-4ce5-4b28-a8de-c4414c06b791 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:39:18.991698 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2cf7a6-4ce5-4b28-a8de-c4414c06b791 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:39:18.993728 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2cf7a6-4ce5-4b28-a8de-c4414c06b791 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:39:19.004844 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 935/1860] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:39:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:39:28.999996 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=6 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:39:36.670686 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d02e1ac-7eb5-4a67-ad0f-fbb237cfb6fd service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:39:36.672132 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5d02e1ac-7eb5-4a67-ad0f-fbb237cfb6fd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:39:36.683642 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d02e1ac-7eb5-4a67-ad0f-fbb237cfb6fd service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:39:36.687197 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2179b9d9-82e7-4129-af4f-118535ca58b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:39:36.687799 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2179b9d9-82e7-4129-af4f-118535ca58b9 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:39:36.688119 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2179b9d9-82e7-4129-af4f-118535ca58b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:39:36.688394 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2179b9d9-82e7-4129-af4f-118535ca58b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:39:36.690160 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2179b9d9-82e7-4129-af4f-118535ca58b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:39:36.699463 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2179b9d9-82e7-4129-af4f-118535ca58b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:39:36.701028 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2179b9d9-82e7-4129-af4f-118535ca58b9 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:39:36.709159 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 926/1861] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:39:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:39:46.705437 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=6 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:39:51.965581 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2cf7a6-4ce5-4b28-a8de-c4414c06b791 service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:39:51.974982 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2cf7a6-4ce5-4b28-a8de-c4414c06b791 service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:39:51.979277 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cd65eb-d632-4900-b857-b57d21e2fda4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:39:51.979875 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cd65eb-d632-4900-b857-b57d21e2fda4 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:39:51.980197 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cd65eb-d632-4900-b857-b57d21e2fda4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:39:51.980472 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cd65eb-d632-4900-b857-b57d21e2fda4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:39:51.982205 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-c8cd65eb-d632-4900-b857-b57d21e2fda4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:39:52.043656 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cd65eb-d632-4900-b857-b57d21e2fda4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:39:52.045530 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cd65eb-d632-4900-b857-b57d21e2fda4 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:39:52.054852 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 936/1862] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:39:51 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:40:02.050353 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=28 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:40:42.711627 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2179b9d9-82e7-4129-af4f-118535ca58b9 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:40:42.721417 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2179b9d9-82e7-4129-af4f-118535ca58b9 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:40:42.724977 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285f88fb-7f2a-4256-b849-82bc8eedbcdb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:40:42.725574 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f88fb-7f2a-4256-b849-82bc8eedbcdb service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:40:42.725851 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f88fb-7f2a-4256-b849-82bc8eedbcdb service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:40:42.726123 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f88fb-7f2a-4256-b849-82bc8eedbcdb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:40:42.734876 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285f88fb-7f2a-4256-b849-82bc8eedbcdb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:40:42.736324 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285f88fb-7f2a-4256-b849-82bc8eedbcdb service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:40:42.745002 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 927/1863] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:40:42 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:40:52.741240 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=6 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:41:42.081397 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cd65eb-d632-4900-b857-b57d21e2fda4 service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:41:42.185190 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cd65eb-d632-4900-b857-b57d21e2fda4 service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:41:42.189769 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4cb350-7e62-4af5-84f6-3b7a7ad69fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:41:42.190410 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4cb350-7e62-4af5-84f6-3b7a7ad69fdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:41:42.190692 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4cb350-7e62-4af5-84f6-3b7a7ad69fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:41:42.190967 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4cb350-7e62-4af5-84f6-3b7a7ad69fdc admin admin] PATH_INFO: `/v3/users/c44fc29f3a014d8c9ca46903a4ad4211` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:41:42.192136 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4cb350-7e62-4af5-84f6-3b7a7ad69fdc admin admin] RBAC: Authorizing `identity:delete_user(user_id=c44fc29f3a014d8c9ca46903a4ad4211)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:41:42.194052 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4cb350-7e62-4af5-84f6-3b7a7ad69fdc admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:41:42.255783 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-be4cb350-7e62-4af5-84f6-3b7a7ad69fdc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c44fc29f3a014d8c9ca46903a4ad4211'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:41:42.256479 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-be4cb350-7e62-4af5-84f6-3b7a7ad69fdc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c44fc29f3a014d8c9ca46903a4ad4211'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:41:42.257090 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-be4cb350-7e62-4af5-84f6-3b7a7ad69fdc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c44fc29f3a014d8c9ca46903a4ad4211'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:41:42.269685 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-be4cb350-7e62-4af5-84f6-3b7a7ad69fdc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c44fc29f3a014d8c9ca46903a4ad4211'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:41:42.277096 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-be4cb350-7e62-4af5-84f6-3b7a7ad69fdc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c44fc29f3a014d8c9ca46903a4ad4211'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:41:42.290233 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 937/1864] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 03:41:42 2022] DELETE /identity/v3/users/c44fc29f3a014d8c9ca46903a4ad4211 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:41:42.295206 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922a185-5b48-4140-a887-349e3f5facef None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:41:42.295658 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922a185-5b48-4140-a887-349e3f5facef None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:41:42.295751 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922a185-5b48-4140-a887-349e3f5facef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:41:42.344958 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-0922a185-5b48-4140-a887-349e3f5facef None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:41:42.346068 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0922a185-5b48-4140-a887-349e3f5facef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:41:42.403741 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 928/1865] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:41:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:41:42.413644 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4cb350-7e62-4af5-84f6-3b7a7ad69fdc admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:41:42.415051 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-be4cb350-7e62-4af5-84f6-3b7a7ad69fdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:41:42.456080 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:41:42.490712 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4cb350-7e62-4af5-84f6-3b7a7ad69fdc admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:41:42.494398 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74311bd-2051-4c75-8874-5e61449f1831 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:41:42.495007 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74311bd-2051-4c75-8874-5e61449f1831 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:41:42.495290 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74311bd-2051-4c75-8874-5e61449f1831 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:41:42.495567 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74311bd-2051-4c75-8874-5e61449f1831 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:41:42.504863 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74311bd-2051-4c75-8874-5e61449f1831 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:41:42.506289 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74311bd-2051-4c75-8874-5e61449f1831 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:41:42.515656 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 938/1866] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:41:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:41:42.787613 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0922a185-5b48-4140-a887-349e3f5facef None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:41:42.789175 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0922a185-5b48-4140-a887-349e3f5facef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:41:42.801138 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0922a185-5b48-4140-a887-349e3f5facef None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:41:42.806558 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1fc408-07bf-4a25-96c7-beac0c48ca9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:41:42.807305 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1fc408-07bf-4a25-96c7-beac0c48ca9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:41:42.807685 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1fc408-07bf-4a25-96c7-beac0c48ca9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:41:42.808040 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1fc408-07bf-4a25-96c7-beac0c48ca9f admin admin] PATH_INFO: `/v3/projects/0429960aa0454ea38fea17c0302a9a8a` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:41:42.809401 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1fc408-07bf-4a25-96c7-beac0c48ca9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0429960aa0454ea38fea17c0302a9a8a)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:41:42.811396 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1fc408-07bf-4a25-96c7-beac0c48ca9f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:41:42.864910 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5d1fc408-07bf-4a25-96c7-beac0c48ca9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0429960aa0454ea38fea17c0302a9a8a', 'request_id': 'req-5d1fc408-07bf-4a25-96c7-beac0c48ca9f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:41:42.870809 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5d1fc408-07bf-4a25-96c7-beac0c48ca9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0429960aa0454ea38fea17c0302a9a8a', 'request_id': 'req-5d1fc408-07bf-4a25-96c7-beac0c48ca9f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:41:42.879410 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5d1fc408-07bf-4a25-96c7-beac0c48ca9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0429960aa0454ea38fea17c0302a9a8a', 'request_id': 'req-5d1fc408-07bf-4a25-96c7-beac0c48ca9f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:41:42.890462 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5d1fc408-07bf-4a25-96c7-beac0c48ca9f admin admin] The token cache is being invalidate because project 0429960aa0454ea38fea17c0302a9a8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 03:41:42.890745 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5d1fc408-07bf-4a25-96c7-beac0c48ca9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:41:42.891557 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5d1fc408-07bf-4a25-96c7-beac0c48ca9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:41:42.892883 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 929/1867] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 03:41:42 2022] DELETE /identity/v3/projects/0429960aa0454ea38fea17c0302a9a8a => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:41:42.896771 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74311bd-2051-4c75-8874-5e61449f1831 service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:41:42.906954 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74311bd-2051-4c75-8874-5e61449f1831 service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:41:42.964092 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab40ff3-c5ab-4442-b6fc-dc339911b534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:41:42.964727 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab40ff3-c5ab-4442-b6fc-dc339911b534 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:41:42.965571 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab40ff3-c5ab-4442-b6fc-dc339911b534 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:41:42.965846 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab40ff3-c5ab-4442-b6fc-dc339911b534 admin admin] PATH_INFO: `/v3/users/e9d5f64e2ae34034802033194e84f192` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:41:42.966991 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab40ff3-c5ab-4442-b6fc-dc339911b534 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e9d5f64e2ae34034802033194e84f192)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:41:42.968937 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab40ff3-c5ab-4442-b6fc-dc339911b534 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:41:43.029216 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3ab40ff3-c5ab-4442-b6fc-dc339911b534 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e9d5f64e2ae34034802033194e84f192'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:41:43.029885 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3ab40ff3-c5ab-4442-b6fc-dc339911b534 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e9d5f64e2ae34034802033194e84f192'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:41:43.030487 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3ab40ff3-c5ab-4442-b6fc-dc339911b534 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e9d5f64e2ae34034802033194e84f192'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:41:43.043625 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3ab40ff3-c5ab-4442-b6fc-dc339911b534 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e9d5f64e2ae34034802033194e84f192'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:41:43.051288 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-3ab40ff3-c5ab-4442-b6fc-dc339911b534 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e9d5f64e2ae34034802033194e84f192'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:41:43.065352 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 939/1868] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 03:41:42 2022] DELETE /identity/v3/users/e9d5f64e2ae34034802033194e84f192 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:41:43.332373 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1fc408-07bf-4a25-96c7-beac0c48ca9f admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:41:43.333832 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5d1fc408-07bf-4a25-96c7-beac0c48ca9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:41:43.346852 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1fc408-07bf-4a25-96c7-beac0c48ca9f admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:41:43.398146 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:41:43.405559 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f7eb12-e9fe-4b88-a11b-5b98720fccb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:41:43.406922 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f7eb12-e9fe-4b88-a11b-5b98720fccb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:41:43.407208 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f7eb12-e9fe-4b88-a11b-5b98720fccb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:41:43.407496 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f7eb12-e9fe-4b88-a11b-5b98720fccb9 admin admin] PATH_INFO: `/v3/projects/e34b33f691d84080834e8375928b171b` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:41:43.408912 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f7eb12-e9fe-4b88-a11b-5b98720fccb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e34b33f691d84080834e8375928b171b)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:41:43.411061 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f7eb12-e9fe-4b88-a11b-5b98720fccb9 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:41:43.457943 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-50f7eb12-e9fe-4b88-a11b-5b98720fccb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e34b33f691d84080834e8375928b171b', 'request_id': 'req-50f7eb12-e9fe-4b88-a11b-5b98720fccb9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:41:43.463993 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-50f7eb12-e9fe-4b88-a11b-5b98720fccb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e34b33f691d84080834e8375928b171b', 'request_id': 'req-50f7eb12-e9fe-4b88-a11b-5b98720fccb9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:41:43.471610 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-50f7eb12-e9fe-4b88-a11b-5b98720fccb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e34b33f691d84080834e8375928b171b', 'request_id': 'req-50f7eb12-e9fe-4b88-a11b-5b98720fccb9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:41:43.481773 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-50f7eb12-e9fe-4b88-a11b-5b98720fccb9 admin admin] The token cache is being invalidate because project e34b33f691d84080834e8375928b171b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 03:41:43.482056 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-50f7eb12-e9fe-4b88-a11b-5b98720fccb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:41:43.482674 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-50f7eb12-e9fe-4b88-a11b-5b98720fccb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:41:43.484174 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 930/1869] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 03:41:43 2022] DELETE /identity/v3/projects/e34b33f691d84080834e8375928b171b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:41:43.489961 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65cd386-8854-4d2f-a3b0-fa8783d908af None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:41:43.490423 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65cd386-8854-4d2f-a3b0-fa8783d908af None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:41:43.490518 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65cd386-8854-4d2f-a3b0-fa8783d908af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:41:43.539305 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-a65cd386-8854-4d2f-a3b0-fa8783d908af None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:41:43.540527 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a65cd386-8854-4d2f-a3b0-fa8783d908af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:41:43.553580 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:41:43.602112 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 940/1870] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:41:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:41:43.609874 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2666ec-f7f4-4b23-ac59-79faf5eae725 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:41:43.610163 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2666ec-f7f4-4b23-ac59-79faf5eae725 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:41:43.610442 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2666ec-f7f4-4b23-ac59-79faf5eae725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:41:43.630469 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-de2666ec-f7f4-4b23-ac59-79faf5eae725 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:41:43.631664 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-de2666ec-f7f4-4b23-ac59-79faf5eae725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:41:43.640057 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 931/1871] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:41:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:41:43.644105 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65cd386-8854-4d2f-a3b0-fa8783d908af None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:41:43.653654 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65cd386-8854-4d2f-a3b0-fa8783d908af None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:41:43.658314 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52abea17-3040-4fb0-b842-ee338b4615e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:41:43.658856 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52abea17-3040-4fb0-b842-ee338b4615e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:41:43.659135 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52abea17-3040-4fb0-b842-ee338b4615e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:41:43.659409 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52abea17-3040-4fb0-b842-ee338b4615e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:41:43.660091 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52abea17-3040-4fb0-b842-ee338b4615e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:41:43.660395 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52abea17-3040-4fb0-b842-ee338b4615e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:41:43.661967 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52abea17-3040-4fb0-b842-ee338b4615e4 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:41:43.673781 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 941/1872] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 03:41:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:41:43.677741 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2666ec-f7f4-4b23-ac59-79faf5eae725 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:41:43.687018 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2666ec-f7f4-4b23-ac59-79faf5eae725 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:41:43.691617 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e7366-55f4-4974-9fe7-89122b528b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:41:43.692253 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e7366-55f4-4974-9fe7-89122b528b5f admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:41:43.692529 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e7366-55f4-4974-9fe7-89122b528b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:41:43.692802 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45e7366-55f4-4974-9fe7-89122b528b5f admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:41:43.693435 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45e7366-55f4-4974-9fe7-89122b528b5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:41:43.695319 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45e7366-55f4-4974-9fe7-89122b528b5f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:41:43.724255 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 932/1873] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:41:43 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:41:43.728384 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52abea17-3040-4fb0-b842-ee338b4615e4 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:41:43.738566 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52abea17-3040-4fb0-b842-ee338b4615e4 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:41:43.795797 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7043fb97-839f-40e8-9b9e-d48d4ddc660c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:41:43.796428 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043fb97-839f-40e8-9b9e-d48d4ddc660c admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:41:43.796708 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043fb97-839f-40e8-9b9e-d48d4ddc660c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:41:43.796982 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7043fb97-839f-40e8-9b9e-d48d4ddc660c admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:41:43.797618 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7043fb97-839f-40e8-9b9e-d48d4ddc660c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:41:43.799523 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7043fb97-839f-40e8-9b9e-d48d4ddc660c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:41:43.822016 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 942/1874] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:41:43 2022] POST /identity/v3/users => generated 292 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:41:43.823990 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e7366-55f4-4974-9fe7-89122b528b5f admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:41:43.834831 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45e7366-55f4-4974-9fe7-89122b528b5f admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:41:43.839524 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a7d5b2-4146-42bc-9bd6-77da1462ea35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:41:43.840220 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a7d5b2-4146-42bc-9bd6-77da1462ea35 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:41:43.840499 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a7d5b2-4146-42bc-9bd6-77da1462ea35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:41:43.840791 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a7d5b2-4146-42bc-9bd6-77da1462ea35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:41:43.841421 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a7d5b2-4146-42bc-9bd6-77da1462ea35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:41:43.842995 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a7d5b2-4146-42bc-9bd6-77da1462ea35 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:41:43.852760 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 933/1875] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:41:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:41:43.856426 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7043fb97-839f-40e8-9b9e-d48d4ddc660c admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:41:43.866123 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7043fb97-839f-40e8-9b9e-d48d4ddc660c admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:41:43.870728 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26d270b-9137-4a31-a424-07fd59d62a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:41:43.871811 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d270b-9137-4a31-a424-07fd59d62a7a admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:41:43.872145 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d270b-9137-4a31-a424-07fd59d62a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:41:43.872421 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d270b-9137-4a31-a424-07fd59d62a7a admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:41:43.873075 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26d270b-9137-4a31-a424-07fd59d62a7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:41:43.874908 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26d270b-9137-4a31-a424-07fd59d62a7a admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:41:43.886583 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 943/1876] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:41:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:41:43.890965 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a7d5b2-4146-42bc-9bd6-77da1462ea35 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:41:43.900664 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a7d5b2-4146-42bc-9bd6-77da1462ea35 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:41:43.905279 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ee13f-4db6-469c-95d2-3dca9cfa53ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:41:43.905914 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ee13f-4db6-469c-95d2-3dca9cfa53ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:41:43.906214 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ee13f-4db6-469c-95d2-3dca9cfa53ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:41:43.906484 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64ee13f-4db6-469c-95d2-3dca9cfa53ce admin admin] PATH_INFO: `/v3/projects/52db136a694c401db77ad13ce84179c7/users/0105b235ce4146d787dbd15068ecea9b/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:41:43.920990 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64ee13f-4db6-469c-95d2-3dca9cfa53ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=52db136a694c401db77ad13ce84179c7, user_id=0105b235ce4146d787dbd15068ecea9b, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:41:43.923637 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64ee13f-4db6-469c-95d2-3dca9cfa53ce admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:41:43.936815 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 934/1877] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:41:43 2022] PUT /identity/v3/projects/52db136a694c401db77ad13ce84179c7/users/0105b235ce4146d787dbd15068ecea9b/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:41:43.948273 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2830252f-1d32-4539-86bd-cbd337e75a25 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:41:43.948560 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2830252f-1d32-4539-86bd-cbd337e75a25 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:41:43.948859 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2830252f-1d32-4539-86bd-cbd337e75a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:41:43.975964 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-2830252f-1d32-4539-86bd-cbd337e75a25 None None] MFA Rules not processed for user `0105b235ce4146d787dbd15068ecea9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:41:43.977156 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-2830252f-1d32-4539-86bd-cbd337e75a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:41:44.041400 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 944/1878] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:41:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:41:44.052087 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ee13f-4db6-469c-95d2-3dca9cfa53ce admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:41:44.055472 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a64ee13f-4db6-469c-95d2-3dca9cfa53ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:41:44.114228 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64ee13f-4db6-469c-95d2-3dca9cfa53ce admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:41:44.118375 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81427c3-7291-4b4d-91ab-30e1ecc2ece2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:41:44.118993 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81427c3-7291-4b4d-91ab-30e1ecc2ece2 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:41:44.119269 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81427c3-7291-4b4d-91ab-30e1ecc2ece2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:41:44.119539 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81427c3-7291-4b4d-91ab-30e1ecc2ece2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:41:44.130357 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81427c3-7291-4b4d-91ab-30e1ecc2ece2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:41:44.131782 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81427c3-7291-4b4d-91ab-30e1ecc2ece2 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:41:44.141334 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 935/1879] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:41:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:41:54.038497 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=90,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:41:54.137134 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=78,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:42:35.996047 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2830252f-1d32-4539-86bd-cbd337e75a25 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:42:35.997572 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-2830252f-1d32-4539-86bd-cbd337e75a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:42:36.100537 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2830252f-1d32-4539-86bd-cbd337e75a25 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:42:36.106430 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cbe393-1e8e-4c59-a500-b4437417319b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4b9bd741574ddabadd4b7651ebc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:42:36.107091 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cbe393-1e8e-4c59-a500-b4437417319b service placement] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:42:36.107373 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cbe393-1e8e-4c59-a500-b4437417319b service placement] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:42:36.107645 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cbe393-1e8e-4c59-a500-b4437417319b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:42:36.109543 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f1cbe393-1e8e-4c59-a500-b4437417319b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:42:36.121929 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cbe393-1e8e-4c59-a500-b4437417319b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:42:36.123496 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cbe393-1e8e-4c59-a500-b4437417319b service placement] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:42:36.133693 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 945/1880] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:42:35 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:42:46.129303 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=41 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:42:53.186374 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94db2c3-54a9-49a6-a850-54ff2c33fb5e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:42:53.186859 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94db2c3-54a9-49a6-a850-54ff2c33fb5e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:42:53.186960 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94db2c3-54a9-49a6-a850-54ff2c33fb5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:42:53.215594 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-e94db2c3-54a9-49a6-a850-54ff2c33fb5e None None] MFA Rules not processed for user `da7ee607b66e43f3823d9d2faf373974`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:42:53.216819 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e94db2c3-54a9-49a6-a850-54ff2c33fb5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:42:53.276946 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 936/1881] 10.222.0.26 () {58 vars in 967 bytes} [Wed Nov 2 03:42:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:42:53.324728 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cbe393-1e8e-4c59-a500-b4437417319b service placement] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:42:53.326322 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f1cbe393-1e8e-4c59-a500-b4437417319b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:42:53.406909 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cbe393-1e8e-4c59-a500-b4437417319b service placement] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:42:53.411818 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d5cba9-780d-4b7b-830a-cc7352370ee8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5193c40a917b4105ab8b27aead1685f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:42:53.412451 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5cba9-780d-4b7b-830a-cc7352370ee8 service swift] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:42:53.412728 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5cba9-780d-4b7b-830a-cc7352370ee8 service swift] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:42:53.412999 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5cba9-780d-4b7b-830a-cc7352370ee8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:42:53.421479 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d5cba9-780d-4b7b-830a-cc7352370ee8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:42:53.423024 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d5cba9-780d-4b7b-830a-cc7352370ee8 service swift] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:42:53.430914 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 946/1882] 10.222.0.26 () {60 vars in 1384 bytes} [Wed Nov 2 03:42:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 03:42:54.721738 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94db2c3-54a9-49a6-a850-54ff2c33fb5e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:42:54.723598 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e94db2c3-54a9-49a6-a850-54ff2c33fb5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:42:54.767048 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a30d99d-a07f-49dd-a302-c353d82707ee None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:42:54.767606 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a30d99d-a07f-49dd-a302-c353d82707ee None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:42:54.767747 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a30d99d-a07f-49dd-a302-c353d82707ee None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:42:54.769580 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 947/1883] 10.222.0.26 () {54 vars in 912 bytes} [Wed Nov 2 03:42:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Nov 02 03:42:54.775346 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b84c71-76e2-4f48-9ff3-2dd602b01ce4 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:42:54.775939 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b84c71-76e2-4f48-9ff3-2dd602b01ce4 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:42:54.776435 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b84c71-76e2-4f48-9ff3-2dd602b01ce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:42:54.819339 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-c3b84c71-76e2-4f48-9ff3-2dd602b01ce4 None None] MFA Rules not processed for user `9b4b9bd741574ddabadd4b7651ebc885`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:42:54.820865 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-c3b84c71-76e2-4f48-9ff3-2dd602b01ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:42:54.830541 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94db2c3-54a9-49a6-a850-54ff2c33fb5e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:42:54.831491 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 948/1884] 10.222.0.26 () {58 vars in 1009 bytes} [Wed Nov 2 03:42:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:42:54.836502 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb840e26-9478-4d5f-af6b-512778d09a5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:42:54.837124 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb840e26-9478-4d5f-af6b-512778d09a5c service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:42:54.837397 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb840e26-9478-4d5f-af6b-512778d09a5c service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:42:54.837667 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb840e26-9478-4d5f-af6b-512778d09a5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:42:54.839724 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-cb840e26-9478-4d5f-af6b-512778d09a5c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:42:54.853661 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb840e26-9478-4d5f-af6b-512778d09a5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:42:54.855347 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb840e26-9478-4d5f-af6b-512778d09a5c service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:42:54.858291 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b84c71-76e2-4f48-9ff3-2dd602b01ce4 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:42:54.859674 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-c3b84c71-76e2-4f48-9ff3-2dd602b01ce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:42:54.871233 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 937/1885] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:42:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:42:54.874154 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b84c71-76e2-4f48-9ff3-2dd602b01ce4 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:42:54.880054 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6a84e6-32f9-4cee-9f2a-5ed0a613af05 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4b9bd741574ddabadd4b7651ebc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:42:54.880749 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6a84e6-32f9-4cee-9f2a-5ed0a613af05 service placement] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:42:54.881099 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6a84e6-32f9-4cee-9f2a-5ed0a613af05 service placement] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:42:54.881441 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6a84e6-32f9-4cee-9f2a-5ed0a613af05 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:42:54.891418 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6a84e6-32f9-4cee-9f2a-5ed0a613af05 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:42:54.893247 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6a84e6-32f9-4cee-9f2a-5ed0a613af05 service placement] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:42:54.905790 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 949/1886] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:42:54 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 03:42:57.133424 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e6e42-1db0-4920-903b-e559e0528f41 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:42:57.133898 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e6e42-1db0-4920-903b-e559e0528f41 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:42:57.133995 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3e6e42-1db0-4920-903b-e559e0528f41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:42:57.153219 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-ba3e6e42-1db0-4920-903b-e559e0528f41 None None] MFA Rules not processed for user `f16c6a2cd8594037ad47475ed1b3161c`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:42:57.154484 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ba3e6e42-1db0-4920-903b-e559e0528f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:42:57.203694 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:42:57.211266 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 938/1887] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:42:57 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:42:57.515075 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6a84e6-32f9-4cee-9f2a-5ed0a613af05 service placement] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:42:57.516537 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-7b6a84e6-32f9-4cee-9f2a-5ed0a613af05 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:42:57.602925 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6a84e6-32f9-4cee-9f2a-5ed0a613af05 service placement] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:42:57.607148 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f45447-e4a7-4c16-8a17-f1e59ee34301 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:42:57.607753 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f45447-e4a7-4c16-8a17-f1e59ee34301 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:42:57.608056 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f45447-e4a7-4c16-8a17-f1e59ee34301 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:42:57.608330 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f45447-e4a7-4c16-8a17-f1e59ee34301 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:42:57.610040 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-03f45447-e4a7-4c16-8a17-f1e59ee34301 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:42:57.620233 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f45447-e4a7-4c16-8a17-f1e59ee34301 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:42:57.621692 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f45447-e4a7-4c16-8a17-f1e59ee34301 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:42:57.630150 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 950/1888] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:42:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:42:58.137359 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3e6e42-1db0-4920-903b-e559e0528f41 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:42:58.152167 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3e6e42-1db0-4920-903b-e559e0528f41 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:42:58.157272 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbf3cc6-93f8-45da-9c26-f82213168b5c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4b9bd741574ddabadd4b7651ebc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:42:58.157896 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf3cc6-93f8-45da-9c26-f82213168b5c service placement] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:42:58.158178 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf3cc6-93f8-45da-9c26-f82213168b5c service placement] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:42:58.158522 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf3cc6-93f8-45da-9c26-f82213168b5c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:42:58.160254 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-dcbf3cc6-93f8-45da-9c26-f82213168b5c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:42:58.172759 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbf3cc6-93f8-45da-9c26-f82213168b5c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:42:58.174343 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbf3cc6-93f8-45da-9c26-f82213168b5c service placement] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:42:58.186534 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 939/1889] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:42:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:43:00.885404 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f45447-e4a7-4c16-8a17-f1e59ee34301 service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:43:00.901447 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f45447-e4a7-4c16-8a17-f1e59ee34301 service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:43:00.907555 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeb60c8-8a14-48d3-a51d-dac2517b60ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:43:00.908197 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb60c8-8a14-48d3-a51d-dac2517b60ea service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:43:00.908490 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb60c8-8a14-48d3-a51d-dac2517b60ea service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:43:00.908755 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeb60c8-8a14-48d3-a51d-dac2517b60ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:43:00.910521 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-4aeb60c8-8a14-48d3-a51d-dac2517b60ea service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:43:00.923538 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeb60c8-8a14-48d3-a51d-dac2517b60ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:43:00.925237 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeb60c8-8a14-48d3-a51d-dac2517b60ea service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:43:00.940392 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 951/1890] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:43:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 03:43:05.768772 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbf3cc6-93f8-45da-9c26-f82213168b5c service placement] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:43:05.782466 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbf3cc6-93f8-45da-9c26-f82213168b5c service placement] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:43:05.788415 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367d2a8c-902c-4983-b95c-a03fa6869f38 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:43:05.789033 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d2a8c-902c-4983-b95c-a03fa6869f38 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:43:05.789312 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d2a8c-902c-4983-b95c-a03fa6869f38 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:43:05.789587 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367d2a8c-902c-4983-b95c-a03fa6869f38 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:43:05.791366 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-367d2a8c-902c-4983-b95c-a03fa6869f38 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:43:05.803677 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367d2a8c-902c-4983-b95c-a03fa6869f38 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:43:05.805273 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367d2a8c-902c-4983-b95c-a03fa6869f38 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:43:05.815727 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 940/1891] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:43:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:43:10.934931 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=88 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:43:15.811239 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=12 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:43:41.514525 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeb60c8-8a14-48d3-a51d-dac2517b60ea service nova] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:43:41.533729 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeb60c8-8a14-48d3-a51d-dac2517b60ea service nova] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:43:41.537405 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25e4fc2-d7eb-4eab-833d-a936fa3dc1f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:43:41.538008 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25e4fc2-d7eb-4eab-833d-a936fa3dc1f5 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:43:41.538287 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25e4fc2-d7eb-4eab-833d-a936fa3dc1f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:43:41.538561 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25e4fc2-d7eb-4eab-833d-a936fa3dc1f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:43:41.540392 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f25e4fc2-d7eb-4eab-833d-a936fa3dc1f5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:43:41.621025 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25e4fc2-d7eb-4eab-833d-a936fa3dc1f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:43:41.622457 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25e4fc2-d7eb-4eab-833d-a936fa3dc1f5 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:43:41.640643 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 952/1892] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:43:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:43:45.654515 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367d2a8c-902c-4983-b95c-a03fa6869f38 service nova] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:43:45.655975 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-367d2a8c-902c-4983-b95c-a03fa6869f38 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:43:45.670318 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367d2a8c-902c-4983-b95c-a03fa6869f38 service nova] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:43:45.675803 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e07df4-ce8c-4545-a4c0-edc2ed5c8bfb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:43:45.676428 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e07df4-ce8c-4545-a4c0-edc2ed5c8bfb service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:43:45.676708 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e07df4-ce8c-4545-a4c0-edc2ed5c8bfb service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:43:45.676982 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e07df4-ce8c-4545-a4c0-edc2ed5c8bfb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:43:45.678647 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e4e07df4-ce8c-4545-a4c0-edc2ed5c8bfb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:43:45.710324 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e07df4-ce8c-4545-a4c0-edc2ed5c8bfb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:43:45.711892 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e07df4-ce8c-4545-a4c0-edc2ed5c8bfb service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:43:45.729484 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 941/1893] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:43:45 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:43:47.401719 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25e4fc2-d7eb-4eab-833d-a936fa3dc1f5 service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:43:47.410315 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25e4fc2-d7eb-4eab-833d-a936fa3dc1f5 service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:43:47.413934 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40471f22-68e7-45c8-b565-70021d33b304 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:43:47.414563 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40471f22-68e7-45c8-b565-70021d33b304 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:43:47.414841 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40471f22-68e7-45c8-b565-70021d33b304 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:43:47.415110 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40471f22-68e7-45c8-b565-70021d33b304 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:43:47.416835 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-40471f22-68e7-45c8-b565-70021d33b304 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:43:47.496718 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40471f22-68e7-45c8-b565-70021d33b304 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:43:47.498190 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40471f22-68e7-45c8-b565-70021d33b304 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:43:47.516796 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 953/1894] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:43:47 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:43:49.355637 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e07df4-ce8c-4545-a4c0-edc2ed5c8bfb service nova] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:43:49.369573 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e07df4-ce8c-4545-a4c0-edc2ed5c8bfb service nova] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:43:49.376389 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd21557-8052-4831-a971-dacf710ae754 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:43:49.376992 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd21557-8052-4831-a971-dacf710ae754 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:43:49.377318 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd21557-8052-4831-a971-dacf710ae754 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:43:49.377558 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd21557-8052-4831-a971-dacf710ae754 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:43:49.379288 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-cfd21557-8052-4831-a971-dacf710ae754 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:43:49.460434 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd21557-8052-4831-a971-dacf710ae754 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:43:49.462013 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd21557-8052-4831-a971-dacf710ae754 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:43:49.485236 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 942/1895] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:43:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:43:57.514096 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=86 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:43:59.477673 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=60 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:44:38.735635 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40471f22-68e7-45c8-b565-70021d33b304 service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:44:38.737229 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-40471f22-68e7-45c8-b565-70021d33b304 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:44:38.748675 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40471f22-68e7-45c8-b565-70021d33b304 service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:44:38.752436 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939a38e6-d311-43f1-8d10-064d6458a170 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:44:38.753033 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a38e6-d311-43f1-8d10-064d6458a170 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:44:38.753393 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a38e6-d311-43f1-8d10-064d6458a170 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:44:38.753665 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939a38e6-d311-43f1-8d10-064d6458a170 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:44:38.755287 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-939a38e6-d311-43f1-8d10-064d6458a170 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:44:38.764221 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939a38e6-d311-43f1-8d10-064d6458a170 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:44:38.765661 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939a38e6-d311-43f1-8d10-064d6458a170 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:44:38.773966 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 954/1896] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:44:38 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:44:48.770486 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=6 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:44:51.919598 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd21557-8052-4831-a971-dacf710ae754 service nova] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:44:51.928952 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd21557-8052-4831-a971-dacf710ae754 service nova] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:44:51.932612 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baacea8-78b3-42f6-8f54-50595a9451dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:44:51.933232 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baacea8-78b3-42f6-8f54-50595a9451dc service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:44:51.933603 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baacea8-78b3-42f6-8f54-50595a9451dc service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:44:51.933877 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baacea8-78b3-42f6-8f54-50595a9451dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:44:51.935594 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8baacea8-78b3-42f6-8f54-50595a9451dc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:44:52.015285 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baacea8-78b3-42f6-8f54-50595a9451dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:44:52.016727 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8baacea8-78b3-42f6-8f54-50595a9451dc service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:44:52.035116 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 943/1897] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:44:51 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:45:02.032526 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=41 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:46:44.370531 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939a38e6-d311-43f1-8d10-064d6458a170 service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:46:44.371934 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-939a38e6-d311-43f1-8d10-064d6458a170 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:46:44.395571 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939a38e6-d311-43f1-8d10-064d6458a170 service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:46:44.399890 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa316894-380d-4e66-8d62-9d2ad4848acd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:46:44.400560 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa316894-380d-4e66-8d62-9d2ad4848acd service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:46:44.400839 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa316894-380d-4e66-8d62-9d2ad4848acd service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:46:44.401112 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa316894-380d-4e66-8d62-9d2ad4848acd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:46:44.409493 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa316894-380d-4e66-8d62-9d2ad4848acd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:46:44.410942 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa316894-380d-4e66-8d62-9d2ad4848acd service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:46:44.419705 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 955/1898] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:46:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:46:54.415787 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=11 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:47:37.986967 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baacea8-78b3-42f6-8f54-50595a9451dc service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:47:37.998351 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baacea8-78b3-42f6-8f54-50595a9451dc service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:47:38.003544 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac310228-ede1-46c5-9cd4-18462a172195 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4b9bd741574ddabadd4b7651ebc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:47:38.004168 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac310228-ede1-46c5-9cd4-18462a172195 service placement] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:47:38.004453 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac310228-ede1-46c5-9cd4-18462a172195 service placement] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:47:38.004729 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac310228-ede1-46c5-9cd4-18462a172195 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:47:38.013817 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac310228-ede1-46c5-9cd4-18462a172195 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:47:38.015419 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac310228-ede1-46c5-9cd4-18462a172195 service placement] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:47:38.025375 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 944/1899] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:47:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:47:40.642693 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa316894-380d-4e66-8d62-9d2ad4848acd service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:47:40.651859 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa316894-380d-4e66-8d62-9d2ad4848acd service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:47:40.655436 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12199ef9-667e-4211-8f45-e981a7f2f619 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:47:40.656057 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12199ef9-667e-4211-8f45-e981a7f2f619 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:47:40.656339 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12199ef9-667e-4211-8f45-e981a7f2f619 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:47:40.656613 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12199ef9-667e-4211-8f45-e981a7f2f619 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:47:40.658320 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-12199ef9-667e-4211-8f45-e981a7f2f619 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:47:40.671343 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12199ef9-667e-4211-8f45-e981a7f2f619 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:47:40.672780 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12199ef9-667e-4211-8f45-e981a7f2f619 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:47:40.684308 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 956/1900] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:47:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:47:44.797656 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac310228-ede1-46c5-9cd4-18462a172195 service placement] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:47:44.857760 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac310228-ede1-46c5-9cd4-18462a172195 service placement] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:47:44.862267 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a469feea-e4b9-48d8-972e-c1302ee55d67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:47:44.862882 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a469feea-e4b9-48d8-972e-c1302ee55d67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:47:44.863158 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a469feea-e4b9-48d8-972e-c1302ee55d67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:47:44.863452 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a469feea-e4b9-48d8-972e-c1302ee55d67 admin admin] PATH_INFO: `/v3/users/0105b235ce4146d787dbd15068ecea9b` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:47:44.864559 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a469feea-e4b9-48d8-972e-c1302ee55d67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0105b235ce4146d787dbd15068ecea9b)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:47:44.866426 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a469feea-e4b9-48d8-972e-c1302ee55d67 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:47:44.911126 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a469feea-e4b9-48d8-972e-c1302ee55d67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0105b235ce4146d787dbd15068ecea9b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:47:44.918537 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a469feea-e4b9-48d8-972e-c1302ee55d67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0105b235ce4146d787dbd15068ecea9b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:47:44.919197 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a469feea-e4b9-48d8-972e-c1302ee55d67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0105b235ce4146d787dbd15068ecea9b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:47:44.930939 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a469feea-e4b9-48d8-972e-c1302ee55d67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0105b235ce4146d787dbd15068ecea9b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:47:44.938621 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a469feea-e4b9-48d8-972e-c1302ee55d67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0105b235ce4146d787dbd15068ecea9b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:47:44.944570 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 945/1901] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 03:47:44 2022] DELETE /identity/v3/users/0105b235ce4146d787dbd15068ecea9b => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:47:44.949062 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e252c91-87ca-4e39-98ea-d25804b068c3 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:47:44.949561 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e252c91-87ca-4e39-98ea-d25804b068c3 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:47:44.949665 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e252c91-87ca-4e39-98ea-d25804b068c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:47:44.969041 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-9e252c91-87ca-4e39-98ea-d25804b068c3 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:47:44.970170 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-9e252c91-87ca-4e39-98ea-d25804b068c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:47:45.028420 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 957/1902] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:47:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:47:45.037934 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a469feea-e4b9-48d8-972e-c1302ee55d67 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:47:45.039369 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a469feea-e4b9-48d8-972e-c1302ee55d67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:47:45.096664 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a469feea-e4b9-48d8-972e-c1302ee55d67 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:47:45.100360 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5465a90-ef48-447f-8dbb-644d09b8e368 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:47:45.100957 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5465a90-ef48-447f-8dbb-644d09b8e368 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:47:45.101240 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5465a90-ef48-447f-8dbb-644d09b8e368 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:47:45.101515 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5465a90-ef48-447f-8dbb-644d09b8e368 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:47:45.110975 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5465a90-ef48-447f-8dbb-644d09b8e368 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:47:45.112437 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5465a90-ef48-447f-8dbb-644d09b8e368 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:47:45.121893 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 946/1903] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:47:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:47:45.418209 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e252c91-87ca-4e39-98ea-d25804b068c3 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:47:45.420073 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-9e252c91-87ca-4e39-98ea-d25804b068c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:47:45.432085 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e252c91-87ca-4e39-98ea-d25804b068c3 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:47:45.437491 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c399fb-7aa5-470d-ac60-0c22dd119f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:47:45.438130 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c399fb-7aa5-470d-ac60-0c22dd119f88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:47:45.438432 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c399fb-7aa5-470d-ac60-0c22dd119f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:47:45.438707 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c399fb-7aa5-470d-ac60-0c22dd119f88 admin admin] PATH_INFO: `/v3/projects/52db136a694c401db77ad13ce84179c7` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:47:45.439836 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c399fb-7aa5-470d-ac60-0c22dd119f88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52db136a694c401db77ad13ce84179c7)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:47:45.441769 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c399fb-7aa5-470d-ac60-0c22dd119f88 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:47:45.495837 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-91c399fb-7aa5-470d-ac60-0c22dd119f88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52db136a694c401db77ad13ce84179c7', 'request_id': 'req-91c399fb-7aa5-470d-ac60-0c22dd119f88'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:47:45.501621 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-91c399fb-7aa5-470d-ac60-0c22dd119f88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52db136a694c401db77ad13ce84179c7', 'request_id': 'req-91c399fb-7aa5-470d-ac60-0c22dd119f88'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:47:45.508302 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-91c399fb-7aa5-470d-ac60-0c22dd119f88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52db136a694c401db77ad13ce84179c7', 'request_id': 'req-91c399fb-7aa5-470d-ac60-0c22dd119f88'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:47:45.525985 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-91c399fb-7aa5-470d-ac60-0c22dd119f88 admin admin] The token cache is being invalidate because project 52db136a694c401db77ad13ce84179c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 03:47:45.526383 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-91c399fb-7aa5-470d-ac60-0c22dd119f88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:47:45.527099 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-91c399fb-7aa5-470d-ac60-0c22dd119f88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:47:45.528702 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 958/1904] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 03:47:45 2022] DELETE /identity/v3/projects/52db136a694c401db77ad13ce84179c7 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:47:45.535114 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1012a5fe-fea6-452c-8c16-56a134f790f6 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:47:45.535403 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1012a5fe-fea6-452c-8c16-56a134f790f6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:47:45.535677 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1012a5fe-fea6-452c-8c16-56a134f790f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:47:45.556964 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-1012a5fe-fea6-452c-8c16-56a134f790f6 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:47:45.558725 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1012a5fe-fea6-452c-8c16-56a134f790f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:47:45.562846 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:47:45.618579 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 947/1905] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:47:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 03:47:45.627194 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d215b293-652e-43d3-8345-8471ff18c85e None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:47:45.627506 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d215b293-652e-43d3-8345-8471ff18c85e None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:47:45.627780 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d215b293-652e-43d3-8345-8471ff18c85e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:47:45.648155 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-d215b293-652e-43d3-8345-8471ff18c85e None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:47:45.649354 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d215b293-652e-43d3-8345-8471ff18c85e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:47:45.658272 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 959/1906] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:47:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:47:45.662301 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1012a5fe-fea6-452c-8c16-56a134f790f6 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:47:45.672605 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1012a5fe-fea6-452c-8c16-56a134f790f6 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:47:45.677542 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aafa3c-34da-4a9a-bbd2-9ebe995f7894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:47:45.678145 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aafa3c-34da-4a9a-bbd2-9ebe995f7894 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:47:45.678425 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aafa3c-34da-4a9a-bbd2-9ebe995f7894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:47:45.678695 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aafa3c-34da-4a9a-bbd2-9ebe995f7894 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:47:45.679333 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aafa3c-34da-4a9a-bbd2-9ebe995f7894 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:47:45.679630 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aafa3c-34da-4a9a-bbd2-9ebe995f7894 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:47:45.681314 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aafa3c-34da-4a9a-bbd2-9ebe995f7894 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:47:45.693710 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 948/1907] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 03:47:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:47:45.697742 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d215b293-652e-43d3-8345-8471ff18c85e None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:47:45.707342 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d215b293-652e-43d3-8345-8471ff18c85e None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:47:45.712072 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d1303d-c83e-43c4-9a0a-d0342eae03f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:47:45.712674 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d1303d-c83e-43c4-9a0a-d0342eae03f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:47:45.712953 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d1303d-c83e-43c4-9a0a-d0342eae03f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:47:45.713225 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d1303d-c83e-43c4-9a0a-d0342eae03f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:47:45.713890 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d1303d-c83e-43c4-9a0a-d0342eae03f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:47:45.715879 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d1303d-c83e-43c4-9a0a-d0342eae03f6 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:47:45.756192 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 960/1908] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:47:45 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:47:45.762128 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aafa3c-34da-4a9a-bbd2-9ebe995f7894 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:47:45.772313 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aafa3c-34da-4a9a-bbd2-9ebe995f7894 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:47:45.826663 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3a1e32-e3b7-4af6-bcc8-d8b171de9404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:47:45.827288 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3a1e32-e3b7-4af6-bcc8-d8b171de9404 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:47:45.827514 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3a1e32-e3b7-4af6-bcc8-d8b171de9404 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:47:45.827782 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3a1e32-e3b7-4af6-bcc8-d8b171de9404 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:47:45.828428 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3a1e32-e3b7-4af6-bcc8-d8b171de9404 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:47:45.830317 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3a1e32-e3b7-4af6-bcc8-d8b171de9404 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:47:45.851496 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 949/1909] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:47:45 2022] POST /identity/v3/users => generated 287 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:47:45.856271 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d1303d-c83e-43c4-9a0a-d0342eae03f6 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:47:45.866570 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d1303d-c83e-43c4-9a0a-d0342eae03f6 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:47:45.871109 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6963331-0d1f-478f-ad5b-e4c789dea7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:47:45.871706 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6963331-0d1f-478f-ad5b-e4c789dea7a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:47:45.872014 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6963331-0d1f-478f-ad5b-e4c789dea7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:47:45.872288 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6963331-0d1f-478f-ad5b-e4c789dea7a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:47:45.872926 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6963331-0d1f-478f-ad5b-e4c789dea7a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:47:45.875089 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6963331-0d1f-478f-ad5b-e4c789dea7a8 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:47:45.884754 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 961/1910] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:47:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:47:45.888655 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3a1e32-e3b7-4af6-bcc8-d8b171de9404 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:47:45.898189 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3a1e32-e3b7-4af6-bcc8-d8b171de9404 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:47:45.903244 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bb05b6-b611-4cca-b82a-099a1b097717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:47:45.903828 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb05b6-b611-4cca-b82a-099a1b097717 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:47:45.904136 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb05b6-b611-4cca-b82a-099a1b097717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:47:45.904404 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bb05b6-b611-4cca-b82a-099a1b097717 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:47:45.905031 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bb05b6-b611-4cca-b82a-099a1b097717 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:47:45.906588 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bb05b6-b611-4cca-b82a-099a1b097717 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:47:45.916923 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 950/1911] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:47:45 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:47:45.921364 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6963331-0d1f-478f-ad5b-e4c789dea7a8 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:47:45.930973 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6963331-0d1f-478f-ad5b-e4c789dea7a8 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:47:45.935604 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f9523-12c3-42a2-8928-cc2198d105d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:47:45.936262 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f9523-12c3-42a2-8928-cc2198d105d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:47:45.936541 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f9523-12c3-42a2-8928-cc2198d105d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:47:45.936811 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169f9523-12c3-42a2-8928-cc2198d105d4 admin admin] PATH_INFO: `/v3/projects/0db59e238ab64fc6b1fccb6cb2e250a0/users/ca57d3cb72a34d71aa6c185b153dc075/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:47:45.951030 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169f9523-12c3-42a2-8928-cc2198d105d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0db59e238ab64fc6b1fccb6cb2e250a0, user_id=ca57d3cb72a34d71aa6c185b153dc075, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:47:45.953690 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169f9523-12c3-42a2-8928-cc2198d105d4 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:47:45.962063 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:47:45.968891 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 962/1912] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:47:45 2022] PUT /identity/v3/projects/0db59e238ab64fc6b1fccb6cb2e250a0/users/ca57d3cb72a34d71aa6c185b153dc075/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 03:47:45.979389 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c09129-6653-4ef0-bbcf-decbdacb375d None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:47:45.980301 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c09129-6653-4ef0-bbcf-decbdacb375d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:47:45.980580 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c09129-6653-4ef0-bbcf-decbdacb375d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:47:46.008121 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-34c09129-6653-4ef0-bbcf-decbdacb375d None None] MFA Rules not processed for user `ca57d3cb72a34d71aa6c185b153dc075`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:47:46.009297 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-34c09129-6653-4ef0-bbcf-decbdacb375d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:47:46.072648 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 951/1913] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:47:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:47:46.083151 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f9523-12c3-42a2-8928-cc2198d105d4 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:47:46.084861 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-169f9523-12c3-42a2-8928-cc2198d105d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:47:46.141443 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169f9523-12c3-42a2-8928-cc2198d105d4 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:47:46.145112 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c72496-ab7d-4efd-81cd-ba6fafcf2dd7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:47:46.145712 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c72496-ab7d-4efd-81cd-ba6fafcf2dd7 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:47:46.145996 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c72496-ab7d-4efd-81cd-ba6fafcf2dd7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:47:46.146275 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c72496-ab7d-4efd-81cd-ba6fafcf2dd7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:47:46.155276 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c72496-ab7d-4efd-81cd-ba6fafcf2dd7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:47:46.156734 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c72496-ab7d-4efd-81cd-ba6fafcf2dd7 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:47:46.165395 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 963/1914] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:47:46 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:47:56.069532 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=95,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:47:56.161520 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=27 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:48:42.486295 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c09129-6653-4ef0-bbcf-decbdacb375d None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:48:42.495920 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c09129-6653-4ef0-bbcf-decbdacb375d None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:48:42.499406 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0016580-aa67-4e1a-bd4d-2fe7f1d8f39f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:48:42.500300 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0016580-aa67-4e1a-bd4d-2fe7f1d8f39f service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:48:42.500572 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0016580-aa67-4e1a-bd4d-2fe7f1d8f39f service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:48:42.500838 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0016580-aa67-4e1a-bd4d-2fe7f1d8f39f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:48:42.502507 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f0016580-aa67-4e1a-bd4d-2fe7f1d8f39f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:48:42.562982 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0016580-aa67-4e1a-bd4d-2fe7f1d8f39f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:48:42.564478 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0016580-aa67-4e1a-bd4d-2fe7f1d8f39f service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:48:42.573608 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 952/1915] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:48:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:48:45.794011 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c72496-ab7d-4efd-81cd-ba6fafcf2dd7 service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:48:45.802411 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c72496-ab7d-4efd-81cd-ba6fafcf2dd7 service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:48:45.806103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fab360-8206-4b56-a701-ef6a811480f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:48:45.806697 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fab360-8206-4b56-a701-ef6a811480f8 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:48:45.806976 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fab360-8206-4b56-a701-ef6a811480f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:48:45.807248 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fab360-8206-4b56-a701-ef6a811480f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:48:45.808963 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-22fab360-8206-4b56-a701-ef6a811480f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:48:45.867306 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fab360-8206-4b56-a701-ef6a811480f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:48:45.868744 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fab360-8206-4b56-a701-ef6a811480f8 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:48:45.878422 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 964/1916] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:48:45 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:48:48.066067 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0016580-aa67-4e1a-bd4d-2fe7f1d8f39f service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:48:48.074650 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0016580-aa67-4e1a-bd4d-2fe7f1d8f39f service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:48:48.078329 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44420596-1d96-4e87-ae00-9e3dcf8d7315 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:48:48.078925 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44420596-1d96-4e87-ae00-9e3dcf8d7315 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:48:48.079225 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44420596-1d96-4e87-ae00-9e3dcf8d7315 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:48:48.079508 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44420596-1d96-4e87-ae00-9e3dcf8d7315 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:48:48.081234 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-44420596-1d96-4e87-ae00-9e3dcf8d7315 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:48:48.138922 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44420596-1d96-4e87-ae00-9e3dcf8d7315 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:48:48.140392 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44420596-1d96-4e87-ae00-9e3dcf8d7315 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:48:48.149253 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 953/1917] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:48:48 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:48:48.876504 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fab360-8206-4b56-a701-ef6a811480f8 service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:48:48.878481 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-22fab360-8206-4b56-a701-ef6a811480f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:48:48.954596 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fab360-8206-4b56-a701-ef6a811480f8 service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:48:48.960730 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad1a6f-19c0-41f4-b8e1-13c6be77e74b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:48:48.961336 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad1a6f-19c0-41f4-b8e1-13c6be77e74b service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:48:48.961651 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad1a6f-19c0-41f4-b8e1-13c6be77e74b service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:48:48.961877 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad1a6f-19c0-41f4-b8e1-13c6be77e74b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:48:48.963671 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-08ad1a6f-19c0-41f4-b8e1-13c6be77e74b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:48:49.022731 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ad1a6f-19c0-41f4-b8e1-13c6be77e74b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:48:49.024363 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ad1a6f-19c0-41f4-b8e1-13c6be77e74b service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:48:49.033349 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 965/1918] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:48:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:48:58.145425 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=56 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:48:59.029073 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=84 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:49:23.949197 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44420596-1d96-4e87-ae00-9e3dcf8d7315 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:49:23.950627 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-44420596-1d96-4e87-ae00-9e3dcf8d7315 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:49:24.009562 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44420596-1d96-4e87-ae00-9e3dcf8d7315 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:49:24.013202 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e5463e-4f54-4da2-acbf-354746428cf3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:49:24.013810 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e5463e-4f54-4da2-acbf-354746428cf3 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:49:24.014087 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e5463e-4f54-4da2-acbf-354746428cf3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:49:24.014429 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e5463e-4f54-4da2-acbf-354746428cf3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:49:24.016271 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-10e5463e-4f54-4da2-acbf-354746428cf3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:49:24.030206 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e5463e-4f54-4da2-acbf-354746428cf3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:49:24.031613 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10e5463e-4f54-4da2-acbf-354746428cf3 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:49:24.042481 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 954/1919] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:49:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:49:34.037649 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=26 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:49:36.282070 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad1a6f-19c0-41f4-b8e1-13c6be77e74b service nova] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:49:36.295273 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad1a6f-19c0-41f4-b8e1-13c6be77e74b service nova] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:49:36.301246 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bab805a-86ec-4d7b-be6c-fd64a23cbf16 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:49:36.301933 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bab805a-86ec-4d7b-be6c-fd64a23cbf16 service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:49:36.302280 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bab805a-86ec-4d7b-be6c-fd64a23cbf16 service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:49:36.302620 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bab805a-86ec-4d7b-be6c-fd64a23cbf16 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:49:36.305652 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-0bab805a-86ec-4d7b-be6c-fd64a23cbf16 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:49:36.321210 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bab805a-86ec-4d7b-be6c-fd64a23cbf16 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:49:36.323770 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bab805a-86ec-4d7b-be6c-fd64a23cbf16 service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:49:36.338689 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 966/1920] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:49:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:49:41.683915 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e5463e-4f54-4da2-acbf-354746428cf3 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:49:41.685482 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-10e5463e-4f54-4da2-acbf-354746428cf3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:49:41.695972 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e5463e-4f54-4da2-acbf-354746428cf3 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:49:41.699553 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6b5ec5-0937-482d-a8f0-9ea15af0217a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:49:41.700181 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6b5ec5-0937-482d-a8f0-9ea15af0217a service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:49:41.700461 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6b5ec5-0937-482d-a8f0-9ea15af0217a service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:49:41.700737 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6b5ec5-0937-482d-a8f0-9ea15af0217a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:49:41.702375 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-bd6b5ec5-0937-482d-a8f0-9ea15af0217a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:49:41.712076 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6b5ec5-0937-482d-a8f0-9ea15af0217a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:49:41.713497 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd6b5ec5-0937-482d-a8f0-9ea15af0217a service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:49:41.722411 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 955/1921] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:49:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:49:46.330867 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=6 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:49:51.718247 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=6 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:49:52.288575 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bab805a-86ec-4d7b-be6c-fd64a23cbf16 service nova] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:49:52.300718 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bab805a-86ec-4d7b-be6c-fd64a23cbf16 service nova] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:49:52.306193 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342664e8-3f04-44f8-80be-4e4ea75df461 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:49:52.306794 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342664e8-3f04-44f8-80be-4e4ea75df461 service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:49:52.307072 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342664e8-3f04-44f8-80be-4e4ea75df461 service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:49:52.307347 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342664e8-3f04-44f8-80be-4e4ea75df461 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:49:52.309289 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-342664e8-3f04-44f8-80be-4e4ea75df461 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:49:52.366927 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342664e8-3f04-44f8-80be-4e4ea75df461 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:49:52.368532 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342664e8-3f04-44f8-80be-4e4ea75df461 service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:49:52.377609 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 967/1922] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:49:52 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:50:02.373219 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=28 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:52:40.685553 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6b5ec5-0937-482d-a8f0-9ea15af0217a service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:52:40.704292 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6b5ec5-0937-482d-a8f0-9ea15af0217a service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:52:40.707930 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6b2a49-5e1e-465b-b883-e2f01e86e630 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:52:40.708551 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b2a49-5e1e-465b-b883-e2f01e86e630 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:52:40.708851 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b2a49-5e1e-465b-b883-e2f01e86e630 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:52:40.709125 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6b2a49-5e1e-465b-b883-e2f01e86e630 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:52:40.710843 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-be6b2a49-5e1e-465b-b883-e2f01e86e630 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:52:40.763745 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6b2a49-5e1e-465b-b883-e2f01e86e630 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:52:40.765193 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6b2a49-5e1e-465b-b883-e2f01e86e630 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:52:40.775938 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 956/1923] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:52:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 03:52:41.006085 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342664e8-3f04-44f8-80be-4e4ea75df461 service nova] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:52:41.007629 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-342664e8-3f04-44f8-80be-4e4ea75df461 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:52:41.135843 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342664e8-3f04-44f8-80be-4e4ea75df461 service nova] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:52:41.140876 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a1ef1b-5ce2-4226-81db-17465b94f702 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4b9bd741574ddabadd4b7651ebc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:52:41.141482 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a1ef1b-5ce2-4226-81db-17465b94f702 service placement] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:52:41.141765 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a1ef1b-5ce2-4226-81db-17465b94f702 service placement] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:52:41.142040 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a1ef1b-5ce2-4226-81db-17465b94f702 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:52:41.143750 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-81a1ef1b-5ce2-4226-81db-17465b94f702 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:52:41.154864 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a1ef1b-5ce2-4226-81db-17465b94f702 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:52:41.156467 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a1ef1b-5ce2-4226-81db-17465b94f702 service placement] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:52:41.166619 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 968/1924] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:52:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:52:50.771137 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=28 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:52:51.162025 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=59 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:52:55.788539 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6b2a49-5e1e-465b-b883-e2f01e86e630 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:52:55.798298 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6b2a49-5e1e-465b-b883-e2f01e86e630 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:52:55.801869 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e809f8-bca8-425f-8a78-881a2819adc6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:52:55.802466 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e809f8-bca8-425f-8a78-881a2819adc6 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:52:55.802740 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e809f8-bca8-425f-8a78-881a2819adc6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:52:55.803010 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e809f8-bca8-425f-8a78-881a2819adc6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:52:55.811246 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e809f8-bca8-425f-8a78-881a2819adc6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:52:55.812673 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e809f8-bca8-425f-8a78-881a2819adc6 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:52:55.821252 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 957/1925] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:52:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:53:05.817459 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=6 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:53:42.405103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a1ef1b-5ce2-4226-81db-17465b94f702 service placement] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:53:42.406770 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-81a1ef1b-5ce2-4226-81db-17465b94f702 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:53:42.449909 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a1ef1b-5ce2-4226-81db-17465b94f702 service placement] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:53:42.455366 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d0acbf-33b0-4436-a520-354a71756238 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:53:42.455978 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0acbf-33b0-4436-a520-354a71756238 service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:53:42.456258 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0acbf-33b0-4436-a520-354a71756238 service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:53:42.456526 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d0acbf-33b0-4436-a520-354a71756238 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:53:42.464613 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d0acbf-33b0-4436-a520-354a71756238 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:53:42.466170 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d0acbf-33b0-4436-a520-354a71756238 service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:53:42.492382 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 969/1926] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:53:42 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:53:52.489825 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=25 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:54:23.348770 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e809f8-bca8-425f-8a78-881a2819adc6 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:54:23.367404 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e809f8-bca8-425f-8a78-881a2819adc6 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:54:23.370997 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f2bb40-e494-42eb-8378-481406191a94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:54:23.371585 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2bb40-e494-42eb-8378-481406191a94 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:54:23.371877 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2bb40-e494-42eb-8378-481406191a94 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:54:23.372174 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f2bb40-e494-42eb-8378-481406191a94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:54:23.382240 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f2bb40-e494-42eb-8378-481406191a94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:54:23.383631 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f2bb40-e494-42eb-8378-481406191a94 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:54:23.394713 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 958/1927] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:54:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:54:33.390037 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=10 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:54:48.618586 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d0acbf-33b0-4436-a520-354a71756238 service nova] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:54:48.628293 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d0acbf-33b0-4436-a520-354a71756238 service nova] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:54:48.631938 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9f1298-7ec6-4454-9ded-92cc1cc3e9ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:54:48.632567 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9f1298-7ec6-4454-9ded-92cc1cc3e9ff service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:54:48.632850 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9f1298-7ec6-4454-9ded-92cc1cc3e9ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:54:48.633125 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9f1298-7ec6-4454-9ded-92cc1cc3e9ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:54:48.640919 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9f1298-7ec6-4454-9ded-92cc1cc3e9ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:54:48.642348 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9f1298-7ec6-4454-9ded-92cc1cc3e9ff service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:54:48.650125 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 970/1928] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:54:48 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:54:56.117351 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f2bb40-e494-42eb-8378-481406191a94 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:54:56.137629 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f2bb40-e494-42eb-8378-481406191a94 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:54:56.141667 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a47d84f-69f8-4ba8-ba1c-d51632e59a37 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:54:56.142250 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a47d84f-69f8-4ba8-ba1c-d51632e59a37 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:54:56.142522 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a47d84f-69f8-4ba8-ba1c-d51632e59a37 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:54:56.142787 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a47d84f-69f8-4ba8-ba1c-d51632e59a37 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:54:56.151211 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a47d84f-69f8-4ba8-ba1c-d51632e59a37 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:54:56.152914 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a47d84f-69f8-4ba8-ba1c-d51632e59a37 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:54:56.178705 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 959/1929] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:54:56 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:54:56.255658 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9f1298-7ec6-4454-9ded-92cc1cc3e9ff service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:54:56.264696 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9f1298-7ec6-4454-9ded-92cc1cc3e9ff service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:54:56.268243 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a4daa9-ff0e-4821-a64f-4df0ba4e7480 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:54:56.268838 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a4daa9-ff0e-4821-a64f-4df0ba4e7480 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:54:56.269114 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a4daa9-ff0e-4821-a64f-4df0ba4e7480 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:54:56.269377 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2a4daa9-ff0e-4821-a64f-4df0ba4e7480 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:54:56.277550 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a4daa9-ff0e-4821-a64f-4df0ba4e7480 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:54:56.278941 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2a4daa9-ff0e-4821-a64f-4df0ba4e7480 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:54:56.305775 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 971/1930] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:54:56 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:54:56.346956 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a47d84f-69f8-4ba8-ba1c-d51632e59a37 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:54:56.356566 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a47d84f-69f8-4ba8-ba1c-d51632e59a37 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:54:56.360221 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b849f747-c724-41ca-aa19-cddc0d786c21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:54:56.360762 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b849f747-c724-41ca-aa19-cddc0d786c21 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:54:56.361036 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b849f747-c724-41ca-aa19-cddc0d786c21 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:54:56.361303 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b849f747-c724-41ca-aa19-cddc0d786c21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:54:56.370402 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b849f747-c724-41ca-aa19-cddc0d786c21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:54:56.371797 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b849f747-c724-41ca-aa19-cddc0d786c21 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:54:56.399656 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 960/1931] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:54:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:54:56.446739 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a4daa9-ff0e-4821-a64f-4df0ba4e7480 service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:54:56.455256 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2a4daa9-ff0e-4821-a64f-4df0ba4e7480 service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:54:56.458851 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a90ad-3334-423a-a97c-7b85dc7f52d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:54:56.459497 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a90ad-3334-423a-a97c-7b85dc7f52d9 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:54:56.459719 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a90ad-3334-423a-a97c-7b85dc7f52d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:54:56.460036 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431a90ad-3334-423a-a97c-7b85dc7f52d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:54:56.468705 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431a90ad-3334-423a-a97c-7b85dc7f52d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:54:56.470119 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431a90ad-3334-423a-a97c-7b85dc7f52d9 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:54:56.498276 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 972/1932] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:54:56 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:54:58.156014 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b849f747-c724-41ca-aa19-cddc0d786c21 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:54:58.173019 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b849f747-c724-41ca-aa19-cddc0d786c21 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:54:58.184345 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c0cc33-05a4-4f72-8464-82eab9e93dad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:54:58.185173 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c0cc33-05a4-4f72-8464-82eab9e93dad service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:54:58.185524 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c0cc33-05a4-4f72-8464-82eab9e93dad service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:54:58.185832 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c0cc33-05a4-4f72-8464-82eab9e93dad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:54:58.196371 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c0cc33-05a4-4f72-8464-82eab9e93dad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:54:58.198102 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c0cc33-05a4-4f72-8464-82eab9e93dad service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:54:58.209777 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 961/1933] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:54:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:55:01.679157 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a90ad-3334-423a-a97c-7b85dc7f52d9 service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:55:01.688961 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431a90ad-3334-423a-a97c-7b85dc7f52d9 service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:55:01.692619 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aeec0b-a661-46dc-8913-2e4d8c4f1171 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:55:01.693263 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aeec0b-a661-46dc-8913-2e4d8c4f1171 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:01.693546 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aeec0b-a661-46dc-8913-2e4d8c4f1171 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:01.693822 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5aeec0b-a661-46dc-8913-2e4d8c4f1171 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:01.695547 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a5aeec0b-a661-46dc-8913-2e4d8c4f1171 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:55:01.778862 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aeec0b-a661-46dc-8913-2e4d8c4f1171 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:55:01.780364 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5aeec0b-a661-46dc-8913-2e4d8c4f1171 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:55:01.798945 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 973/1934] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:55:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:55:05.102462 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c0cc33-05a4-4f72-8464-82eab9e93dad service nova] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:55:05.119524 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c0cc33-05a4-4f72-8464-82eab9e93dad service nova] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:55:05.139244 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26db2c86-71ef-4056-a439-ad15ffd6b937 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4b9bd741574ddabadd4b7651ebc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:55:05.140347 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db2c86-71ef-4056-a439-ad15ffd6b937 service placement] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:05.141567 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db2c86-71ef-4056-a439-ad15ffd6b937 service placement] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:05.142569 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26db2c86-71ef-4056-a439-ad15ffd6b937 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:05.144796 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-26db2c86-71ef-4056-a439-ad15ffd6b937 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:55:05.159402 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26db2c86-71ef-4056-a439-ad15ffd6b937 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:55:05.163222 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26db2c86-71ef-4056-a439-ad15ffd6b937 service placement] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:55:05.170366 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aeec0b-a661-46dc-8913-2e4d8c4f1171 service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:55:05.172535 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a5aeec0b-a661-46dc-8913-2e4d8c4f1171 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:55:05.175564 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 962/1935] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:55:05 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:55:05.190738 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5aeec0b-a661-46dc-8913-2e4d8c4f1171 service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:55:05.195363 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece5d178-35f7-454c-82ed-9a6de262415d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:55:05.196029 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece5d178-35f7-454c-82ed-9a6de262415d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:05.196572 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece5d178-35f7-454c-82ed-9a6de262415d admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:05.196572 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece5d178-35f7-454c-82ed-9a6de262415d admin admin] PATH_INFO: `/v3/users/eed64456ec5c42b08df447e9a537c4d0` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:05.214936 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece5d178-35f7-454c-82ed-9a6de262415d admin admin] RBAC: Authorizing `identity:delete_user(user_id=eed64456ec5c42b08df447e9a537c4d0)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:55:05.216910 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece5d178-35f7-454c-82ed-9a6de262415d admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:55:05.257843 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:55:05.264616 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ece5d178-35f7-454c-82ed-9a6de262415d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eed64456ec5c42b08df447e9a537c4d0'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:55:05.265412 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ece5d178-35f7-454c-82ed-9a6de262415d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eed64456ec5c42b08df447e9a537c4d0'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:55:05.266020 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ece5d178-35f7-454c-82ed-9a6de262415d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eed64456ec5c42b08df447e9a537c4d0'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:55:05.277800 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ece5d178-35f7-454c-82ed-9a6de262415d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eed64456ec5c42b08df447e9a537c4d0'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:55:05.285088 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ece5d178-35f7-454c-82ed-9a6de262415d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eed64456ec5c42b08df447e9a537c4d0'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:55:05.297698 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 974/1936] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 03:55:05 2022] DELETE /identity/v3/users/eed64456ec5c42b08df447e9a537c4d0 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 03:55:05.471013 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26db2c86-71ef-4056-a439-ad15ffd6b937 service placement] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:55:05.485346 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26db2c86-71ef-4056-a439-ad15ffd6b937 service placement] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:55:05.543770 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7bfc2a-de2a-4a67-8b46-1bcadd50adce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:55:05.544445 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7bfc2a-de2a-4a67-8b46-1bcadd50adce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:05.544724 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7bfc2a-de2a-4a67-8b46-1bcadd50adce admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:05.544996 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7bfc2a-de2a-4a67-8b46-1bcadd50adce admin admin] PATH_INFO: `/v3/projects/c3a4704c59a141a2a1abbb5abb65c994` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:05.546100 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7bfc2a-de2a-4a67-8b46-1bcadd50adce admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3a4704c59a141a2a1abbb5abb65c994)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:55:05.548014 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7bfc2a-de2a-4a67-8b46-1bcadd50adce admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:55:05.592663 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1a7bfc2a-de2a-4a67-8b46-1bcadd50adce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3a4704c59a141a2a1abbb5abb65c994', 'request_id': 'req-1a7bfc2a-de2a-4a67-8b46-1bcadd50adce'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:55:05.598585 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1a7bfc2a-de2a-4a67-8b46-1bcadd50adce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3a4704c59a141a2a1abbb5abb65c994', 'request_id': 'req-1a7bfc2a-de2a-4a67-8b46-1bcadd50adce'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:55:05.606225 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1a7bfc2a-de2a-4a67-8b46-1bcadd50adce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3a4704c59a141a2a1abbb5abb65c994', 'request_id': 'req-1a7bfc2a-de2a-4a67-8b46-1bcadd50adce'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:55:05.612352 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:55:05.617354 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1a7bfc2a-de2a-4a67-8b46-1bcadd50adce admin admin] The token cache is being invalidate because project c3a4704c59a141a2a1abbb5abb65c994 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 03:55:05.617878 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1a7bfc2a-de2a-4a67-8b46-1bcadd50adce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:55:05.619008 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1a7bfc2a-de2a-4a67-8b46-1bcadd50adce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:55:05.620838 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 963/1937] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 03:55:05 2022] DELETE /identity/v3/projects/c3a4704c59a141a2a1abbb5abb65c994 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:55:05.627305 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc5b5a0-6034-4b82-978b-1ae4db0b1df3 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:05.627590 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc5b5a0-6034-4b82-978b-1ae4db0b1df3 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:05.627862 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc5b5a0-6034-4b82-978b-1ae4db0b1df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:05.684816 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-6fc5b5a0-6034-4b82-978b-1ae4db0b1df3 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:55:05.686032 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-6fc5b5a0-6034-4b82-978b-1ae4db0b1df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:55:05.744092 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 975/1938] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:55:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:55:05.752317 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec222c70-6338-494d-b667-69fb619cb4d9 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:05.752604 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec222c70-6338-494d-b667-69fb619cb4d9 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:05.752880 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec222c70-6338-494d-b667-69fb619cb4d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:05.772297 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-ec222c70-6338-494d-b667-69fb619cb4d9 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:55:05.773496 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ec222c70-6338-494d-b667-69fb619cb4d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:55:05.781940 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 964/1939] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:55:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:55:05.785934 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc5b5a0-6034-4b82-978b-1ae4db0b1df3 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:55:05.796026 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc5b5a0-6034-4b82-978b-1ae4db0b1df3 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:55:05.800516 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d6c804-1e1b-46cf-bd91-5597dd234d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:55:05.801110 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d6c804-1e1b-46cf-bd91-5597dd234d89 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:05.801387 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d6c804-1e1b-46cf-bd91-5597dd234d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:05.801656 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d6c804-1e1b-46cf-bd91-5597dd234d89 admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:05.802291 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d6c804-1e1b-46cf-bd91-5597dd234d89 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:55:05.802590 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d6c804-1e1b-46cf-bd91-5597dd234d89 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:55:05.804189 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d6c804-1e1b-46cf-bd91-5597dd234d89 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:55:05.815453 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 976/1940] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 03:55:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:55:05.819606 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec222c70-6338-494d-b667-69fb619cb4d9 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:55:05.828733 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec222c70-6338-494d-b667-69fb619cb4d9 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:55:05.833299 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c5aed4-5b19-49af-88fd-56684d304fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:55:05.833894 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c5aed4-5b19-49af-88fd-56684d304fa8 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:05.834167 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c5aed4-5b19-49af-88fd-56684d304fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:05.834432 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c5aed4-5b19-49af-88fd-56684d304fa8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:05.835049 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c5aed4-5b19-49af-88fd-56684d304fa8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:55:05.836972 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c5aed4-5b19-49af-88fd-56684d304fa8 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:55:05.864213 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 965/1941] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:55:05 2022] POST /identity/v3/projects => generated 403 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:55:05.868272 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d6c804-1e1b-46cf-bd91-5597dd234d89 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:55:05.878184 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d6c804-1e1b-46cf-bd91-5597dd234d89 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:55:05.933519 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ed78da-88ae-4be3-a9b6-b655d1ebc28b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:55:05.934110 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ed78da-88ae-4be3-a9b6-b655d1ebc28b admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:05.934387 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ed78da-88ae-4be3-a9b6-b655d1ebc28b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:05.934656 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ed78da-88ae-4be3-a9b6-b655d1ebc28b admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:05.935280 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ed78da-88ae-4be3-a9b6-b655d1ebc28b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:55:05.937201 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ed78da-88ae-4be3-a9b6-b655d1ebc28b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:55:05.957878 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 977/1942] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:55:05 2022] POST /identity/v3/users => generated 302 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:55:05.962151 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c5aed4-5b19-49af-88fd-56684d304fa8 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:55:05.972793 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c5aed4-5b19-49af-88fd-56684d304fa8 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:55:05.977378 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f8b04a-cdd0-40d6-b708-f03c337722b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:55:05.977968 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8b04a-cdd0-40d6-b708-f03c337722b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:05.978241 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8b04a-cdd0-40d6-b708-f03c337722b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:05.978515 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f8b04a-cdd0-40d6-b708-f03c337722b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:05.979147 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f8b04a-cdd0-40d6-b708-f03c337722b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:55:05.980831 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f8b04a-cdd0-40d6-b708-f03c337722b8 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:55:05.990856 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 966/1943] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:55:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:55:05.994745 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ed78da-88ae-4be3-a9b6-b655d1ebc28b admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:55:05.999934 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:55:06.006067 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ed78da-88ae-4be3-a9b6-b655d1ebc28b admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:55:06.012453 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dec0a4-cd76-4e9a-b043-2a7155f486f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:55:06.013170 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dec0a4-cd76-4e9a-b043-2a7155f486f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:06.013455 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dec0a4-cd76-4e9a-b043-2a7155f486f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:06.013729 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dec0a4-cd76-4e9a-b043-2a7155f486f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:06.014376 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dec0a4-cd76-4e9a-b043-2a7155f486f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:55:06.015977 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dec0a4-cd76-4e9a-b043-2a7155f486f1 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:55:06.027312 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 978/1944] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:55:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 03:55:06.031405 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f8b04a-cdd0-40d6-b708-f03c337722b8 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:55:06.041356 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f8b04a-cdd0-40d6-b708-f03c337722b8 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:55:06.045941 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98336ac0-ca0e-4e46-97a3-7f972351ee53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:55:06.046578 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98336ac0-ca0e-4e46-97a3-7f972351ee53 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:06.046858 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98336ac0-ca0e-4e46-97a3-7f972351ee53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:06.047134 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98336ac0-ca0e-4e46-97a3-7f972351ee53 admin admin] PATH_INFO: `/v3/projects/50f753b323234052863bf89826ecfbeb/users/b805b7dd0e49444cb1b76e006d778b8c/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:06.061587 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98336ac0-ca0e-4e46-97a3-7f972351ee53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50f753b323234052863bf89826ecfbeb, user_id=b805b7dd0e49444cb1b76e006d778b8c, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:55:06.064256 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98336ac0-ca0e-4e46-97a3-7f972351ee53 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:55:06.077471 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 967/1945] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:55:06 2022] PUT /identity/v3/projects/50f753b323234052863bf89826ecfbeb/users/b805b7dd0e49444cb1b76e006d778b8c/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:55:06.082689 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43f722-9845-4ee3-bc18-504b38e486c8 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:06.082972 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43f722-9845-4ee3-bc18-504b38e486c8 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:06.083244 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f43f722-9845-4ee3-bc18-504b38e486c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:06.103447 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-4f43f722-9845-4ee3-bc18-504b38e486c8 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:55:06.104718 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-4f43f722-9845-4ee3-bc18-504b38e486c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:55:06.162840 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 979/1946] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:55:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:55:06.172423 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98336ac0-ca0e-4e46-97a3-7f972351ee53 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:55:06.174407 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-98336ac0-ca0e-4e46-97a3-7f972351ee53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:55:06.243265 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98336ac0-ca0e-4e46-97a3-7f972351ee53 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:55:06.247002 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2f541a-595b-45de-beaa-bc631c9b583a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:55:06.247548 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f541a-595b-45de-beaa-bc631c9b583a service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:06.247829 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f541a-595b-45de-beaa-bc631c9b583a service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:06.248145 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2f541a-595b-45de-beaa-bc631c9b583a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:06.257379 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2f541a-595b-45de-beaa-bc631c9b583a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:55:06.258791 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2f541a-595b-45de-beaa-bc631c9b583a service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:55:06.268788 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 968/1947] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:55:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:55:06.895716 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3dd2b7-5d5e-4e2e-b1bc-0f264a75ecdc None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:06.896290 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3dd2b7-5d5e-4e2e-b1bc-0f264a75ecdc None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:06.896290 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3dd2b7-5d5e-4e2e-b1bc-0f264a75ecdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:06.924274 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-ce3dd2b7-5d5e-4e2e-b1bc-0f264a75ecdc None None] MFA Rules not processed for user `b805b7dd0e49444cb1b76e006d778b8c`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:55:06.925479 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ce3dd2b7-5d5e-4e2e-b1bc-0f264a75ecdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:55:06.992492 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 980/1948] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:55:06 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:55:06.997597 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2f541a-595b-45de-beaa-bc631c9b583a service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:55:07.007771 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2f541a-595b-45de-beaa-bc631c9b583a service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:55:07.012528 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4af7ba-1991-414e-a078-bbc1a316151a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:55:07.013204 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4af7ba-1991-414e-a078-bbc1a316151a admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:07.013592 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4af7ba-1991-414e-a078-bbc1a316151a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:07.013872 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4af7ba-1991-414e-a078-bbc1a316151a admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:07.014508 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4af7ba-1991-414e-a078-bbc1a316151a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:55:07.016437 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4af7ba-1991-414e-a078-bbc1a316151a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:55:07.046390 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 969/1949] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:55:06 2022] POST /identity/v3/projects => generated 403 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:55:07.050315 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3dd2b7-5d5e-4e2e-b1bc-0f264a75ecdc None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:55:07.061153 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3dd2b7-5d5e-4e2e-b1bc-0f264a75ecdc None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:55:07.120227 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d3b71b-6811-4e47-aa0c-2423d609c7a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:55:07.120833 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d3b71b-6811-4e47-aa0c-2423d609c7a2 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:07.121107 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d3b71b-6811-4e47-aa0c-2423d609c7a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:07.121377 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d3b71b-6811-4e47-aa0c-2423d609c7a2 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:07.121998 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d3b71b-6811-4e47-aa0c-2423d609c7a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:55:07.123895 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d3b71b-6811-4e47-aa0c-2423d609c7a2 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:55:07.145070 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 981/1950] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:55:07 2022] POST /identity/v3/users => generated 308 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:55:07.149094 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4af7ba-1991-414e-a078-bbc1a316151a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:55:07.159919 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4af7ba-1991-414e-a078-bbc1a316151a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:55:07.164479 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f2c6ae-7718-4647-abde-7d21e0d8add0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:55:07.165072 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2c6ae-7718-4647-abde-7d21e0d8add0 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:07.165350 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2c6ae-7718-4647-abde-7d21e0d8add0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:07.165621 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f2c6ae-7718-4647-abde-7d21e0d8add0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:07.166255 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f2c6ae-7718-4647-abde-7d21e0d8add0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:55:07.168522 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f2c6ae-7718-4647-abde-7d21e0d8add0 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:55:07.178307 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 970/1951] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:55:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:55:07.182297 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d3b71b-6811-4e47-aa0c-2423d609c7a2 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:55:07.192317 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d3b71b-6811-4e47-aa0c-2423d609c7a2 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:55:07.197137 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c944d42-04ff-414f-912f-8e13a4ffd4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:55:07.197798 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c944d42-04ff-414f-912f-8e13a4ffd4fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:07.198113 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c944d42-04ff-414f-912f-8e13a4ffd4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:07.198388 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c944d42-04ff-414f-912f-8e13a4ffd4fe admin admin] PATH_INFO: `/v3/projects/449f4dfdb489407eb76a26e246e2ecbd/users/59218f45648c415a82fa6138afb3b0f4/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:07.206904 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:55:07.215180 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c944d42-04ff-414f-912f-8e13a4ffd4fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=449f4dfdb489407eb76a26e246e2ecbd, user_id=59218f45648c415a82fa6138afb3b0f4, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:55:07.217914 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c944d42-04ff-414f-912f-8e13a4ffd4fe admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:55:07.230311 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 982/1952] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:55:07 2022] PUT /identity/v3/projects/449f4dfdb489407eb76a26e246e2ecbd/users/59218f45648c415a82fa6138afb3b0f4/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:55:07.234253 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f2c6ae-7718-4647-abde-7d21e0d8add0 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:55:07.244400 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f2c6ae-7718-4647-abde-7d21e0d8add0 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:55:07.292163 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:55:07.302076 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4743ee6-b5dd-433d-87f7-46aba91c890b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:55:07.302627 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4743ee6-b5dd-433d-87f7-46aba91c890b admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:07.302907 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4743ee6-b5dd-433d-87f7-46aba91c890b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:07.303179 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4743ee6-b5dd-433d-87f7-46aba91c890b admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:07.303808 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4743ee6-b5dd-433d-87f7-46aba91c890b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:55:07.305430 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4743ee6-b5dd-433d-87f7-46aba91c890b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:55:07.315243 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 971/1953] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:55:07 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:55:07.319461 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c944d42-04ff-414f-912f-8e13a4ffd4fe admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:55:07.329644 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c944d42-04ff-414f-912f-8e13a4ffd4fe admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:55:07.334269 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fde378-260d-4009-91f0-9494c0e0f253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:55:07.334915 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fde378-260d-4009-91f0-9494c0e0f253 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:07.335195 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fde378-260d-4009-91f0-9494c0e0f253 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:07.335467 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fde378-260d-4009-91f0-9494c0e0f253 admin admin] PATH_INFO: `/v3/projects/449f4dfdb489407eb76a26e246e2ecbd/users/59218f45648c415a82fa6138afb3b0f4/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:07.337406 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fde378-260d-4009-91f0-9494c0e0f253 admin admin] RBAC: Authorizing `identity:create_grant(project_id=449f4dfdb489407eb76a26e246e2ecbd, user_id=59218f45648c415a82fa6138afb3b0f4, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:55:07.340617 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fde378-260d-4009-91f0-9494c0e0f253 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:55:07.350575 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 983/1954] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:55:07 2022] PUT /identity/v3/projects/449f4dfdb489407eb76a26e246e2ecbd/users/59218f45648c415a82fa6138afb3b0f4/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:55:07.973235 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa44b30-2904-4d4a-a423-e822c670d028 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:07.973527 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa44b30-2904-4d4a-a423-e822c670d028 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:07.973814 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa44b30-2904-4d4a-a423-e822c670d028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:08.002488 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-7aa44b30-2904-4d4a-a423-e822c670d028 None None] MFA Rules not processed for user `59218f45648c415a82fa6138afb3b0f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:55:08.003709 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7aa44b30-2904-4d4a-a423-e822c670d028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:55:08.074771 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 972/1955] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:55:07 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:55:09.041592 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fde378-260d-4009-91f0-9494c0e0f253 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:55:09.043294 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d8fde378-260d-4009-91f0-9494c0e0f253 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:55:09.103096 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa44b30-2904-4d4a-a423-e822c670d028 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:55:09.124308 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fde378-260d-4009-91f0-9494c0e0f253 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:55:09.130360 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fbfdb9-43d3-4226-87b0-5f85404268c2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4b9bd741574ddabadd4b7651ebc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:55:09.130985 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fbfdb9-43d3-4226-87b0-5f85404268c2 service placement] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:09.131659 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fbfdb9-43d3-4226-87b0-5f85404268c2 service placement] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:09.131659 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fbfdb9-43d3-4226-87b0-5f85404268c2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:09.133420 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f9fbfdb9-43d3-4226-87b0-5f85404268c2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:55:09.146214 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fbfdb9-43d3-4226-87b0-5f85404268c2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:55:09.147798 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fbfdb9-43d3-4226-87b0-5f85404268c2 service placement] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:55:09.158864 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 984/1956] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:55:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:55:09.168039 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa44b30-2904-4d4a-a423-e822c670d028 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:55:09.172694 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcdcff2-9a68-422a-99cb-3a23ab55ec59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:55:09.173472 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcdcff2-9a68-422a-99cb-3a23ab55ec59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:09.173762 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcdcff2-9a68-422a-99cb-3a23ab55ec59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:09.174041 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcdcff2-9a68-422a-99cb-3a23ab55ec59 admin admin] PATH_INFO: `/v3/users/b805b7dd0e49444cb1b76e006d778b8c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:09.175243 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcdcff2-9a68-422a-99cb-3a23ab55ec59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b805b7dd0e49444cb1b76e006d778b8c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:55:09.177214 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dcdcff2-9a68-422a-99cb-3a23ab55ec59 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:55:09.223580 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9dcdcff2-9a68-422a-99cb-3a23ab55ec59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b805b7dd0e49444cb1b76e006d778b8c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:55:09.230340 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9dcdcff2-9a68-422a-99cb-3a23ab55ec59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b805b7dd0e49444cb1b76e006d778b8c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:55:09.230995 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9dcdcff2-9a68-422a-99cb-3a23ab55ec59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b805b7dd0e49444cb1b76e006d778b8c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:55:09.242730 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9dcdcff2-9a68-422a-99cb-3a23ab55ec59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b805b7dd0e49444cb1b76e006d778b8c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:55:09.245334 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:55:09.250768 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9dcdcff2-9a68-422a-99cb-3a23ab55ec59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b805b7dd0e49444cb1b76e006d778b8c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:55:09.257877 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 973/1957] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 03:55:09 2022] DELETE /identity/v3/users/b805b7dd0e49444cb1b76e006d778b8c => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:55:09.435342 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fbfdb9-43d3-4226-87b0-5f85404268c2 service placement] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:55:09.448381 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fbfdb9-43d3-4226-87b0-5f85404268c2 service placement] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:55:09.506099 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458ab464-fb96-4820-9aa4-1ef0ea046a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:55:09.506791 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458ab464-fb96-4820-9aa4-1ef0ea046a72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:09.507075 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458ab464-fb96-4820-9aa4-1ef0ea046a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:09.507349 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458ab464-fb96-4820-9aa4-1ef0ea046a72 admin admin] PATH_INFO: `/v3/projects/50f753b323234052863bf89826ecfbeb` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:09.508499 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458ab464-fb96-4820-9aa4-1ef0ea046a72 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50f753b323234052863bf89826ecfbeb)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:55:09.511014 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458ab464-fb96-4820-9aa4-1ef0ea046a72 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:55:09.556566 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-458ab464-fb96-4820-9aa4-1ef0ea046a72 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50f753b323234052863bf89826ecfbeb', 'request_id': 'req-458ab464-fb96-4820-9aa4-1ef0ea046a72'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:55:09.563379 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-458ab464-fb96-4820-9aa4-1ef0ea046a72 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50f753b323234052863bf89826ecfbeb', 'request_id': 'req-458ab464-fb96-4820-9aa4-1ef0ea046a72'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:55:09.569494 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-458ab464-fb96-4820-9aa4-1ef0ea046a72 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50f753b323234052863bf89826ecfbeb', 'request_id': 'req-458ab464-fb96-4820-9aa4-1ef0ea046a72'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:55:09.572042 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:55:09.583132 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-458ab464-fb96-4820-9aa4-1ef0ea046a72 admin admin] The token cache is being invalidate because project 50f753b323234052863bf89826ecfbeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 03:55:09.583445 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-458ab464-fb96-4820-9aa4-1ef0ea046a72 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:55:09.584099 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-458ab464-fb96-4820-9aa4-1ef0ea046a72 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:55:09.585618 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 985/1958] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 03:55:09 2022] DELETE /identity/v3/projects/50f753b323234052863bf89826ecfbeb => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:55:09.589645 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcdcff2-9a68-422a-99cb-3a23ab55ec59 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:55:09.591068 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9dcdcff2-9a68-422a-99cb-3a23ab55ec59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:55:09.603772 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcdcff2-9a68-422a-99cb-3a23ab55ec59 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:55:09.659180 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11854fd1-2d6c-433e-986a-39841f512d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:55:09.659804 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11854fd1-2d6c-433e-986a-39841f512d2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:09.660118 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11854fd1-2d6c-433e-986a-39841f512d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:09.660390 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11854fd1-2d6c-433e-986a-39841f512d2f admin admin] PATH_INFO: `/v3/users/59218f45648c415a82fa6138afb3b0f4` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:09.661507 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11854fd1-2d6c-433e-986a-39841f512d2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=59218f45648c415a82fa6138afb3b0f4)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:55:09.663435 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11854fd1-2d6c-433e-986a-39841f512d2f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:55:09.709099 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-11854fd1-2d6c-433e-986a-39841f512d2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59218f45648c415a82fa6138afb3b0f4'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:55:09.716038 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-11854fd1-2d6c-433e-986a-39841f512d2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59218f45648c415a82fa6138afb3b0f4'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:55:09.716707 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-11854fd1-2d6c-433e-986a-39841f512d2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59218f45648c415a82fa6138afb3b0f4'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:55:09.728552 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-11854fd1-2d6c-433e-986a-39841f512d2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59218f45648c415a82fa6138afb3b0f4'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:55:09.736388 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-11854fd1-2d6c-433e-986a-39841f512d2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59218f45648c415a82fa6138afb3b0f4'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:55:09.742338 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 974/1959] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 03:55:09 2022] DELETE /identity/v3/users/59218f45648c415a82fa6138afb3b0f4 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:55:09.920911 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458ab464-fb96-4820-9aa4-1ef0ea046a72 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:55:09.922399 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-458ab464-fb96-4820-9aa4-1ef0ea046a72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:55:09.935435 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458ab464-fb96-4820-9aa4-1ef0ea046a72 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:55:09.991105 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8462af-e9e3-43d6-b6fd-db57905956ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:55:09.991885 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8462af-e9e3-43d6-b6fd-db57905956ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:09.992192 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8462af-e9e3-43d6-b6fd-db57905956ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:09.992459 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8462af-e9e3-43d6-b6fd-db57905956ed admin admin] PATH_INFO: `/v3/projects/449f4dfdb489407eb76a26e246e2ecbd` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:09.993562 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8462af-e9e3-43d6-b6fd-db57905956ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=449f4dfdb489407eb76a26e246e2ecbd)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:55:09.996179 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8462af-e9e3-43d6-b6fd-db57905956ed admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:55:10.042541 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5e8462af-e9e3-43d6-b6fd-db57905956ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '449f4dfdb489407eb76a26e246e2ecbd', 'request_id': 'req-5e8462af-e9e3-43d6-b6fd-db57905956ed'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:55:10.047879 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5e8462af-e9e3-43d6-b6fd-db57905956ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '449f4dfdb489407eb76a26e246e2ecbd', 'request_id': 'req-5e8462af-e9e3-43d6-b6fd-db57905956ed'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:55:10.053854 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5e8462af-e9e3-43d6-b6fd-db57905956ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '449f4dfdb489407eb76a26e246e2ecbd', 'request_id': 'req-5e8462af-e9e3-43d6-b6fd-db57905956ed'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:55:10.065846 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5e8462af-e9e3-43d6-b6fd-db57905956ed admin admin] The token cache is being invalidate because project 449f4dfdb489407eb76a26e246e2ecbd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 03:55:10.066129 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5e8462af-e9e3-43d6-b6fd-db57905956ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:55:10.066732 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5e8462af-e9e3-43d6-b6fd-db57905956ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:55:10.068245 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 986/1960] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 03:55:09 2022] DELETE /identity/v3/projects/449f4dfdb489407eb76a26e246e2ecbd => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:55:10.073175 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63440127-3c75-4275-b282-59a3a9064cc9 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:10.073460 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63440127-3c75-4275-b282-59a3a9064cc9 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:10.073739 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63440127-3c75-4275-b282-59a3a9064cc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:10.094011 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-63440127-3c75-4275-b282-59a3a9064cc9 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:55:10.095262 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-63440127-3c75-4275-b282-59a3a9064cc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:55:10.156529 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 975/1961] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:55:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:55:10.164182 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc281e-2d6c-4caa-97ac-a05a936883f4 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:10.164491 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc281e-2d6c-4caa-97ac-a05a936883f4 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:10.164762 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bc281e-2d6c-4caa-97ac-a05a936883f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:10.184477 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-b6bc281e-2d6c-4caa-97ac-a05a936883f4 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:55:10.185663 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-b6bc281e-2d6c-4caa-97ac-a05a936883f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:55:10.193402 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 987/1962] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:55:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:55:10.197364 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63440127-3c75-4275-b282-59a3a9064cc9 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:55:10.207286 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63440127-3c75-4275-b282-59a3a9064cc9 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:55:10.211921 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2780bca-d0d1-4a64-8224-b6f39f05298d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:55:10.212544 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2780bca-d0d1-4a64-8224-b6f39f05298d admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:10.212825 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2780bca-d0d1-4a64-8224-b6f39f05298d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:10.213099 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2780bca-d0d1-4a64-8224-b6f39f05298d admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:10.214248 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2780bca-d0d1-4a64-8224-b6f39f05298d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:55:10.214551 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2780bca-d0d1-4a64-8224-b6f39f05298d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:55:10.216169 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2780bca-d0d1-4a64-8224-b6f39f05298d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:55:10.224720 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:55:10.228510 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 976/1963] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 03:55:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:55:10.233298 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bc281e-2d6c-4caa-97ac-a05a936883f4 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:55:10.242295 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bc281e-2d6c-4caa-97ac-a05a936883f4 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:55:10.246957 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe497404-acb6-4a76-872a-6bbf2c403504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:55:10.247550 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe497404-acb6-4a76-872a-6bbf2c403504 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:10.247825 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe497404-acb6-4a76-872a-6bbf2c403504 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:10.248130 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe497404-acb6-4a76-872a-6bbf2c403504 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:10.248766 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe497404-acb6-4a76-872a-6bbf2c403504 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:55:10.250899 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe497404-acb6-4a76-872a-6bbf2c403504 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:55:10.277401 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 988/1964] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:55:10 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 03:55:10.281418 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2780bca-d0d1-4a64-8224-b6f39f05298d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:55:10.291366 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2780bca-d0d1-4a64-8224-b6f39f05298d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:55:10.347382 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9254ef-e3b7-48fa-bf27-e540a503f70d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:55:10.347896 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9254ef-e3b7-48fa-bf27-e540a503f70d admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:10.348203 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9254ef-e3b7-48fa-bf27-e540a503f70d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:10.348474 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9254ef-e3b7-48fa-bf27-e540a503f70d admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:10.349090 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9254ef-e3b7-48fa-bf27-e540a503f70d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:55:10.350995 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9254ef-e3b7-48fa-bf27-e540a503f70d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:55:10.374908 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 977/1965] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:55:10 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:55:10.376262 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe497404-acb6-4a76-872a-6bbf2c403504 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:55:10.387336 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe497404-acb6-4a76-872a-6bbf2c403504 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:55:10.392958 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16581737-f540-49ba-81df-ba41b8226469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:55:10.393565 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16581737-f540-49ba-81df-ba41b8226469 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:10.393848 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16581737-f540-49ba-81df-ba41b8226469 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:10.394122 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16581737-f540-49ba-81df-ba41b8226469 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:10.394766 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16581737-f540-49ba-81df-ba41b8226469 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:55:10.396400 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16581737-f540-49ba-81df-ba41b8226469 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:55:10.406983 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 989/1966] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:55:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:55:10.410852 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9254ef-e3b7-48fa-bf27-e540a503f70d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:55:10.421263 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9254ef-e3b7-48fa-bf27-e540a503f70d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:55:10.425864 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8627a4f-47dd-414a-b1de-da7de107d536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:55:10.426456 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8627a4f-47dd-414a-b1de-da7de107d536 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:10.426736 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8627a4f-47dd-414a-b1de-da7de107d536 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:10.427006 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8627a4f-47dd-414a-b1de-da7de107d536 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:10.427635 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8627a4f-47dd-414a-b1de-da7de107d536 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:55:10.429267 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8627a4f-47dd-414a-b1de-da7de107d536 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:55:10.439594 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 978/1967] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:55:10 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:55:10.443394 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16581737-f540-49ba-81df-ba41b8226469 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:55:10.453720 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16581737-f540-49ba-81df-ba41b8226469 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:55:10.458296 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa238dc2-6ae2-4b1c-b88a-bdd46a7346da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:55:10.458935 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa238dc2-6ae2-4b1c-b88a-bdd46a7346da admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:10.459214 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa238dc2-6ae2-4b1c-b88a-bdd46a7346da admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:10.459488 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa238dc2-6ae2-4b1c-b88a-bdd46a7346da admin admin] PATH_INFO: `/v3/projects/65c936e5fe464fbe873fd325b75df15c/users/e60638f348384b788807d5aad28d74ad/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:10.474371 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa238dc2-6ae2-4b1c-b88a-bdd46a7346da admin admin] RBAC: Authorizing `identity:create_grant(project_id=65c936e5fe464fbe873fd325b75df15c, user_id=e60638f348384b788807d5aad28d74ad, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:55:10.477053 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa238dc2-6ae2-4b1c-b88a-bdd46a7346da admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:55:10.489819 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 990/1968] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:55:10 2022] PUT /identity/v3/projects/65c936e5fe464fbe873fd325b75df15c/users/e60638f348384b788807d5aad28d74ad/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:55:10.501123 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f663d001-1f71-4363-bcd3-06fcbbe4944e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:10.501412 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f663d001-1f71-4363-bcd3-06fcbbe4944e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:10.501686 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f663d001-1f71-4363-bcd3-06fcbbe4944e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:10.529971 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-f663d001-1f71-4363-bcd3-06fcbbe4944e None None] MFA Rules not processed for user `e60638f348384b788807d5aad28d74ad`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:55:10.531400 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f663d001-1f71-4363-bcd3-06fcbbe4944e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:55:10.594793 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 979/1969] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:55:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:55:10.599463 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa238dc2-6ae2-4b1c-b88a-bdd46a7346da admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:55:10.620256 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:55:10.662860 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa238dc2-6ae2-4b1c-b88a-bdd46a7346da admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:55:10.667630 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7407402-af1d-4b33-9244-4e382d74467f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:55:10.668247 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7407402-af1d-4b33-9244-4e382d74467f admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:10.668522 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7407402-af1d-4b33-9244-4e382d74467f admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:10.668794 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7407402-af1d-4b33-9244-4e382d74467f admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:10.669413 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7407402-af1d-4b33-9244-4e382d74467f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:55:10.671322 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7407402-af1d-4b33-9244-4e382d74467f admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:55:10.696867 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 991/1970] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:55:10 2022] POST /identity/v3/projects => generated 373 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:55:10.701474 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f663d001-1f71-4363-bcd3-06fcbbe4944e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:55:10.711758 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f663d001-1f71-4363-bcd3-06fcbbe4944e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:55:10.766069 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465fa225-6555-40ac-81b4-f27456676591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:55:10.766667 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465fa225-6555-40ac-81b4-f27456676591 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:10.766944 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465fa225-6555-40ac-81b4-f27456676591 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:10.767215 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465fa225-6555-40ac-81b4-f27456676591 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:10.767844 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465fa225-6555-40ac-81b4-f27456676591 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:55:10.769766 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-465fa225-6555-40ac-81b4-f27456676591 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:55:10.785603 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:55:10.792898 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 980/1971] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:55:10 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 03:55:10.796967 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7407402-af1d-4b33-9244-4e382d74467f admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:55:10.807557 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7407402-af1d-4b33-9244-4e382d74467f admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:55:10.812184 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a8f436-d1c6-4160-b7af-13698d317919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:55:10.812782 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a8f436-d1c6-4160-b7af-13698d317919 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:10.813059 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a8f436-d1c6-4160-b7af-13698d317919 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:10.813332 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a8f436-d1c6-4160-b7af-13698d317919 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:10.813965 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a8f436-d1c6-4160-b7af-13698d317919 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:55:10.815547 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a8f436-d1c6-4160-b7af-13698d317919 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:55:10.826689 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 992/1972] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:55:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:55:10.830534 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465fa225-6555-40ac-81b4-f27456676591 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:55:10.840406 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465fa225-6555-40ac-81b4-f27456676591 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:55:10.845055 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9e6cf9-ff13-4bfa-8926-42325685309b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:55:10.845704 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e6cf9-ff13-4bfa-8926-42325685309b admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:10.845984 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e6cf9-ff13-4bfa-8926-42325685309b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:10.846261 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9e6cf9-ff13-4bfa-8926-42325685309b admin admin] PATH_INFO: `/v3/projects/5286b614774c4eb48550cb53f67707eb/users/5b08a5b625d4487683f211c37dfaa88e/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:10.860871 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9e6cf9-ff13-4bfa-8926-42325685309b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5286b614774c4eb48550cb53f67707eb, user_id=5b08a5b625d4487683f211c37dfaa88e, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:55:10.863455 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9e6cf9-ff13-4bfa-8926-42325685309b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:55:10.877109 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 981/1973] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:55:10 2022] PUT /identity/v3/projects/5286b614774c4eb48550cb53f67707eb/users/5b08a5b625d4487683f211c37dfaa88e/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:55:10.881069 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a8f436-d1c6-4160-b7af-13698d317919 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:55:10.891825 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a8f436-d1c6-4160-b7af-13698d317919 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:55:10.953627 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd915871-e6d8-4dab-a68c-9521d5658b64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:55:10.954240 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd915871-e6d8-4dab-a68c-9521d5658b64 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:10.954520 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd915871-e6d8-4dab-a68c-9521d5658b64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:10.954793 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd915871-e6d8-4dab-a68c-9521d5658b64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:10.955431 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd915871-e6d8-4dab-a68c-9521d5658b64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:55:10.957063 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd915871-e6d8-4dab-a68c-9521d5658b64 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:55:10.967487 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 993/1974] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:55:10 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:55:10.971650 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9e6cf9-ff13-4bfa-8926-42325685309b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:55:10.983546 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9e6cf9-ff13-4bfa-8926-42325685309b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:55:10.989146 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28e6843-327a-4e9c-bc19-9be8ee86b3f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:55:10.989666 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28e6843-327a-4e9c-bc19-9be8ee86b3f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:10.990030 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28e6843-327a-4e9c-bc19-9be8ee86b3f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:10.990381 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28e6843-327a-4e9c-bc19-9be8ee86b3f1 admin admin] PATH_INFO: `/v3/projects/5286b614774c4eb48550cb53f67707eb/users/5b08a5b625d4487683f211c37dfaa88e/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:10.992376 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28e6843-327a-4e9c-bc19-9be8ee86b3f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5286b614774c4eb48550cb53f67707eb, user_id=5b08a5b625d4487683f211c37dfaa88e, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:55:10.995744 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28e6843-327a-4e9c-bc19-9be8ee86b3f1 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:55:11.006985 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 982/1975] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:55:10 2022] PUT /identity/v3/projects/5286b614774c4eb48550cb53f67707eb/users/5b08a5b625d4487683f211c37dfaa88e/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:55:11.017733 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc85eb-85c7-411e-9c3a-43446cd76be4 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:11.018048 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc85eb-85c7-411e-9c3a-43446cd76be4 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:11.018328 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc85eb-85c7-411e-9c3a-43446cd76be4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:11.045687 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-76bc85eb-85c7-411e-9c3a-43446cd76be4 None None] MFA Rules not processed for user `5b08a5b625d4487683f211c37dfaa88e`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:55:11.046887 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-76bc85eb-85c7-411e-9c3a-43446cd76be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:55:11.113955 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 994/1976] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:55:11 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:55:21.003073 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=16,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:55:21.111007 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=92,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:55:36.411008 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbccaf2d-f6e4-4f57-9956-4d5dcb4ff077 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:36.411549 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbccaf2d-f6e4-4f57-9956-4d5dcb4ff077 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:36.411549 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbccaf2d-f6e4-4f57-9956-4d5dcb4ff077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:36.450690 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-cbccaf2d-f6e4-4f57-9956-4d5dcb4ff077 None None] MFA Rules not processed for user `7984dc1d452a4c998b44e6d26a299df8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:55:36.451870 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-cbccaf2d-f6e4-4f57-9956-4d5dcb4ff077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:55:36.509775 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 983/1977] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:55:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:55:36.772295 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bc85eb-85c7-411e-9c3a-43446cd76be4 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:55:36.773793 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-76bc85eb-85c7-411e-9c3a-43446cd76be4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:55:36.823518 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbccaf2d-f6e4-4f57-9956-4d5dcb4ff077 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:55:36.825673 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-cbccaf2d-f6e4-4f57-9956-4d5dcb4ff077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:55:36.857348 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bc85eb-85c7-411e-9c3a-43446cd76be4 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:55:36.863110 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed1e565-b153-4fb8-a41a-57a918ad9dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:55:36.863754 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed1e565-b153-4fb8-a41a-57a918ad9dfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:36.864132 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed1e565-b153-4fb8-a41a-57a918ad9dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:36.864703 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed1e565-b153-4fb8-a41a-57a918ad9dfb admin admin] PATH_INFO: `/v3/users/ca57d3cb72a34d71aa6c185b153dc075` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:36.865960 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed1e565-b153-4fb8-a41a-57a918ad9dfb admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca57d3cb72a34d71aa6c185b153dc075)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:55:36.867983 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed1e565-b153-4fb8-a41a-57a918ad9dfb admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:55:36.910303 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbccaf2d-f6e4-4f57-9956-4d5dcb4ff077 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:55:36.914392 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08ab122-5722-41ba-8350-8ec1681f461c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:55:36.915032 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ab122-5722-41ba-8350-8ec1681f461c service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:36.915287 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ab122-5722-41ba-8350-8ec1681f461c service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:36.915557 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08ab122-5722-41ba-8350-8ec1681f461c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:36.917530 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c08ab122-5722-41ba-8350-8ec1681f461c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:55:36.927368 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08ab122-5722-41ba-8350-8ec1681f461c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:55:36.928931 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08ab122-5722-41ba-8350-8ec1681f461c service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:55:36.932309 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-6ed1e565-b153-4fb8-a41a-57a918ad9dfb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca57d3cb72a34d71aa6c185b153dc075'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:55:36.933508 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-6ed1e565-b153-4fb8-a41a-57a918ad9dfb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca57d3cb72a34d71aa6c185b153dc075'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:55:36.933508 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-6ed1e565-b153-4fb8-a41a-57a918ad9dfb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca57d3cb72a34d71aa6c185b153dc075'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:55:36.937271 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 984/1978] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:55:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 03:55:36.938096 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08ab122-5722-41ba-8350-8ec1681f461c service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:55:36.939457 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c08ab122-5722-41ba-8350-8ec1681f461c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:55:36.947065 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-6ed1e565-b153-4fb8-a41a-57a918ad9dfb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca57d3cb72a34d71aa6c185b153dc075'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:55:36.957026 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-6ed1e565-b153-4fb8-a41a-57a918ad9dfb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca57d3cb72a34d71aa6c185b153dc075'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:55:36.971794 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 995/1979] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 03:55:36 2022] DELETE /identity/v3/users/ca57d3cb72a34d71aa6c185b153dc075 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:55:36.976213 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bed148-03e3-4d11-9a5f-5da5ee8d7705 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:36.976502 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bed148-03e3-4d11-9a5f-5da5ee8d7705 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:36.976778 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bed148-03e3-4d11-9a5f-5da5ee8d7705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:36.999504 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08ab122-5722-41ba-8350-8ec1681f461c service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:55:37.001035 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-d1bed148-03e3-4d11-9a5f-5da5ee8d7705 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:55:37.002250 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d1bed148-03e3-4d11-9a5f-5da5ee8d7705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:55:37.053448 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:55:37.061660 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45c48cd-1f94-4ea5-ac6f-46375a66412f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:55:37.062286 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c48cd-1f94-4ea5-ac6f-46375a66412f service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:37.062583 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c48cd-1f94-4ea5-ac6f-46375a66412f service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:37.062850 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c48cd-1f94-4ea5-ac6f-46375a66412f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:37.064747 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a45c48cd-1f94-4ea5-ac6f-46375a66412f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:55:37.069386 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 996/1980] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:55:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:55:37.091447 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bed148-03e3-4d11-9a5f-5da5ee8d7705 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:55:37.093219 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d1bed148-03e3-4d11-9a5f-5da5ee8d7705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:55:37.116917 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:55:37.144471 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45c48cd-1f94-4ea5-ac6f-46375a66412f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:55:37.145916 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45c48cd-1f94-4ea5-ac6f-46375a66412f service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:55:37.156334 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 985/1981] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:55:36 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:55:37.167141 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bed148-03e3-4d11-9a5f-5da5ee8d7705 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:55:37.174812 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1be074c-7b55-4019-a1a5-80046f06c5fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:55:37.175563 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be074c-7b55-4019-a1a5-80046f06c5fc service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:37.175899 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be074c-7b55-4019-a1a5-80046f06c5fc service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:37.176340 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1be074c-7b55-4019-a1a5-80046f06c5fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:37.192998 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1be074c-7b55-4019-a1a5-80046f06c5fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:55:37.194501 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1be074c-7b55-4019-a1a5-80046f06c5fc service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:55:37.205536 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 997/1982] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:55:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:55:37.304415 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45c48cd-1f94-4ea5-ac6f-46375a66412f service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:55:37.306331 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a45c48cd-1f94-4ea5-ac6f-46375a66412f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:55:37.389718 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45c48cd-1f94-4ea5-ac6f-46375a66412f service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:55:37.394814 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804a5053-2b25-4c1f-9bdd-ee7eb0f9d822 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4b9bd741574ddabadd4b7651ebc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:55:37.395437 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804a5053-2b25-4c1f-9bdd-ee7eb0f9d822 service placement] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:37.395717 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804a5053-2b25-4c1f-9bdd-ee7eb0f9d822 service placement] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:37.396049 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804a5053-2b25-4c1f-9bdd-ee7eb0f9d822 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:37.397796 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-804a5053-2b25-4c1f-9bdd-ee7eb0f9d822 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:55:37.410523 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804a5053-2b25-4c1f-9bdd-ee7eb0f9d822 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:55:37.412308 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804a5053-2b25-4c1f-9bdd-ee7eb0f9d822 service placement] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:55:37.423856 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 986/1983] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:55:37 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:55:37.505646 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1be074c-7b55-4019-a1a5-80046f06c5fc service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:55:37.507147 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d1be074c-7b55-4019-a1a5-80046f06c5fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:55:37.521301 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1be074c-7b55-4019-a1a5-80046f06c5fc service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:55:37.526634 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055bd831-0591-4ef8-8edb-333c98c77498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:55:37.527358 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055bd831-0591-4ef8-8edb-333c98c77498 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:37.527827 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055bd831-0591-4ef8-8edb-333c98c77498 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:37.528222 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055bd831-0591-4ef8-8edb-333c98c77498 admin admin] PATH_INFO: `/v3/projects/0db59e238ab64fc6b1fccb6cb2e250a0` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:37.529942 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055bd831-0591-4ef8-8edb-333c98c77498 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0db59e238ab64fc6b1fccb6cb2e250a0)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:55:37.531781 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055bd831-0591-4ef8-8edb-333c98c77498 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:55:37.584464 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-055bd831-0591-4ef8-8edb-333c98c77498 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0db59e238ab64fc6b1fccb6cb2e250a0', 'request_id': 'req-055bd831-0591-4ef8-8edb-333c98c77498'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:55:37.590227 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-055bd831-0591-4ef8-8edb-333c98c77498 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0db59e238ab64fc6b1fccb6cb2e250a0', 'request_id': 'req-055bd831-0591-4ef8-8edb-333c98c77498'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:55:37.596473 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-055bd831-0591-4ef8-8edb-333c98c77498 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0db59e238ab64fc6b1fccb6cb2e250a0', 'request_id': 'req-055bd831-0591-4ef8-8edb-333c98c77498'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:55:37.609554 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-055bd831-0591-4ef8-8edb-333c98c77498 admin admin] The token cache is being invalidate because project 0db59e238ab64fc6b1fccb6cb2e250a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 03:55:37.609982 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-055bd831-0591-4ef8-8edb-333c98c77498 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:55:37.610644 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-055bd831-0591-4ef8-8edb-333c98c77498 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:55:37.612665 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 998/1984] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 03:55:37 2022] DELETE /identity/v3/projects/0db59e238ab64fc6b1fccb6cb2e250a0 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 03:55:37.619211 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c19fb9-ccce-44ed-94ab-5c37a3f82714 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:37.619505 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c19fb9-ccce-44ed-94ab-5c37a3f82714 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:37.619805 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c19fb9-ccce-44ed-94ab-5c37a3f82714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:37.640326 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-f5c19fb9-ccce-44ed-94ab-5c37a3f82714 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:55:37.641438 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f5c19fb9-ccce-44ed-94ab-5c37a3f82714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:55:37.703063 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 987/1985] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:55:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:55:37.711188 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d59d43a-28de-4aa5-b034-9380cddeb1b5 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:37.711473 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d59d43a-28de-4aa5-b034-9380cddeb1b5 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:37.711752 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d59d43a-28de-4aa5-b034-9380cddeb1b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:37.732642 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-0d59d43a-28de-4aa5-b034-9380cddeb1b5 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:55:37.733897 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-0d59d43a-28de-4aa5-b034-9380cddeb1b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:55:37.741841 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 999/1986] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:55:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:55:37.745945 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c19fb9-ccce-44ed-94ab-5c37a3f82714 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:55:37.756027 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c19fb9-ccce-44ed-94ab-5c37a3f82714 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:55:37.760695 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba72e2e-31ca-4b31-bb48-6f42d8df5b7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:55:37.761186 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba72e2e-31ca-4b31-bb48-6f42d8df5b7e admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:37.761465 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba72e2e-31ca-4b31-bb48-6f42d8df5b7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:37.761766 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba72e2e-31ca-4b31-bb48-6f42d8df5b7e admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:37.762408 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba72e2e-31ca-4b31-bb48-6f42d8df5b7e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:55:37.762708 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba72e2e-31ca-4b31-bb48-6f42d8df5b7e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:55:37.764356 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba72e2e-31ca-4b31-bb48-6f42d8df5b7e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:55:37.782885 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 988/1987] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 03:55:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 03:55:37.784394 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d59d43a-28de-4aa5-b034-9380cddeb1b5 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:55:37.796821 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d59d43a-28de-4aa5-b034-9380cddeb1b5 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:55:37.801886 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74a984e-96cc-4992-a799-e3bcbc329ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:55:37.802502 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74a984e-96cc-4992-a799-e3bcbc329ab8 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:37.802783 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74a984e-96cc-4992-a799-e3bcbc329ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:37.803057 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74a984e-96cc-4992-a799-e3bcbc329ab8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:37.803707 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74a984e-96cc-4992-a799-e3bcbc329ab8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:55:37.806064 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74a984e-96cc-4992-a799-e3bcbc329ab8 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:55:37.835490 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1000/1988] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:55:37 2022] POST /identity/v3/projects => generated 397 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:55:37.839696 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba72e2e-31ca-4b31-bb48-6f42d8df5b7e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:55:37.850314 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba72e2e-31ca-4b31-bb48-6f42d8df5b7e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:55:37.862728 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:55:37.910818 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114c21a8-6949-4b87-ac5f-332d1c8154da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:55:37.911428 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c21a8-6949-4b87-ac5f-332d1c8154da admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:37.911706 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c21a8-6949-4b87-ac5f-332d1c8154da admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:37.912009 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c21a8-6949-4b87-ac5f-332d1c8154da admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:37.912655 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114c21a8-6949-4b87-ac5f-332d1c8154da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:55:37.914552 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114c21a8-6949-4b87-ac5f-332d1c8154da admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:55:37.937006 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 989/1989] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:55:37 2022] POST /identity/v3/users => generated 299 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 03:55:37.941128 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74a984e-96cc-4992-a799-e3bcbc329ab8 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:55:37.951464 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74a984e-96cc-4992-a799-e3bcbc329ab8 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:55:37.956338 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df04f935-689d-4f45-abb2-08db2271f11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:55:37.956883 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04f935-689d-4f45-abb2-08db2271f11b admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:37.957161 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04f935-689d-4f45-abb2-08db2271f11b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:37.957452 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04f935-689d-4f45-abb2-08db2271f11b admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:37.958093 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df04f935-689d-4f45-abb2-08db2271f11b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:55:37.959706 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df04f935-689d-4f45-abb2-08db2271f11b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:55:37.972898 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1001/1990] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:55:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:55:37.974134 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114c21a8-6949-4b87-ac5f-332d1c8154da admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:55:37.985366 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114c21a8-6949-4b87-ac5f-332d1c8154da admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:55:37.990092 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c81a4b6-d04b-4666-8211-ed1a5bdd95bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:55:37.990722 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c81a4b6-d04b-4666-8211-ed1a5bdd95bb admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:37.991004 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c81a4b6-d04b-4666-8211-ed1a5bdd95bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:37.991279 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c81a4b6-d04b-4666-8211-ed1a5bdd95bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:37.991918 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c81a4b6-d04b-4666-8211-ed1a5bdd95bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:55:37.993544 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c81a4b6-d04b-4666-8211-ed1a5bdd95bb admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:55:38.004532 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 990/1991] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:55:37 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:55:38.008628 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df04f935-689d-4f45-abb2-08db2271f11b admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:55:38.018841 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df04f935-689d-4f45-abb2-08db2271f11b admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:55:38.024051 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b76ba1-f4b0-4c89-ba8c-edbee9ccd153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:55:38.024721 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b76ba1-f4b0-4c89-ba8c-edbee9ccd153 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:38.024980 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b76ba1-f4b0-4c89-ba8c-edbee9ccd153 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:38.025255 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b76ba1-f4b0-4c89-ba8c-edbee9ccd153 admin admin] PATH_INFO: `/v3/projects/a04e470e9b6343febac95c1b1d9fb47c/users/1151a2dd24604c98accfa0821c0d5b04/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:38.044779 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b76ba1-f4b0-4c89-ba8c-edbee9ccd153 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a04e470e9b6343febac95c1b1d9fb47c, user_id=1151a2dd24604c98accfa0821c0d5b04, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:55:38.047507 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b76ba1-f4b0-4c89-ba8c-edbee9ccd153 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:55:38.060611 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1002/1992] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:55:38 2022] PUT /identity/v3/projects/a04e470e9b6343febac95c1b1d9fb47c/users/1151a2dd24604c98accfa0821c0d5b04/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 03:55:38.072359 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5221be2-e3b3-44a2-98d8-395014e40d5a None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:38.072883 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5221be2-e3b3-44a2-98d8-395014e40d5a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:38.073210 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5221be2-e3b3-44a2-98d8-395014e40d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:38.103043 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a5221be2-e3b3-44a2-98d8-395014e40d5a None None] MFA Rules not processed for user `1151a2dd24604c98accfa0821c0d5b04`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:55:38.104327 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a5221be2-e3b3-44a2-98d8-395014e40d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:55:38.173753 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 991/1993] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:55:38 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:55:46.945134 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b76ba1-f4b0-4c89-ba8c-edbee9ccd153 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:55:46.946936 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-94b76ba1-f4b0-4c89-ba8c-edbee9ccd153 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:55:46.993360 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:55:47.024115 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b76ba1-f4b0-4c89-ba8c-edbee9ccd153 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:55:47.029571 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e480c0-d8b4-4ac1-b04f-4f4f917d8ed7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:55:47.030172 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e480c0-d8b4-4ac1-b04f-4f4f917d8ed7 service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:47.030450 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e480c0-d8b4-4ac1-b04f-4f4f917d8ed7 service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:47.030721 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e480c0-d8b4-4ac1-b04f-4f4f917d8ed7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:47.032478 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-c4e480c0-d8b4-4ac1-b04f-4f4f917d8ed7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:55:47.044234 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e480c0-d8b4-4ac1-b04f-4f4f917d8ed7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:55:47.045776 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e480c0-d8b4-4ac1-b04f-4f4f917d8ed7 service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:55:47.056396 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1003/1994] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:55:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:55:48.170167 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=60,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:55:51.180768 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5221be2-e3b3-44a2-98d8-395014e40d5a None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:55:51.182252 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a5221be2-e3b3-44a2-98d8-395014e40d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:55:51.199234 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5221be2-e3b3-44a2-98d8-395014e40d5a None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:55:51.205551 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace0ddb0-34b2-468d-9b07-47f8f219daab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:55:51.206171 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0ddb0-34b2-468d-9b07-47f8f219daab service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:55:51.206448 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0ddb0-34b2-468d-9b07-47f8f219daab service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:55:51.206721 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace0ddb0-34b2-468d-9b07-47f8f219daab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:55:51.208543 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ace0ddb0-34b2-468d-9b07-47f8f219daab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:55:51.221654 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace0ddb0-34b2-468d-9b07-47f8f219daab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:55:51.223238 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace0ddb0-34b2-468d-9b07-47f8f219daab service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:55:51.234647 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 992/1995] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:55:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:55:57.051904 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=15 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:56:01.229507 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=6 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:56:57.069928 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e480c0-d8b4-4ac1-b04f-4f4f917d8ed7 service nova] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:56:57.071340 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-c4e480c0-d8b4-4ac1-b04f-4f4f917d8ed7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:56:57.140724 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e480c0-d8b4-4ac1-b04f-4f4f917d8ed7 service nova] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:56:57.144394 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757d6301-1a35-4e0b-b87f-795a5f6355a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:56:57.145005 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757d6301-1a35-4e0b-b87f-795a5f6355a8 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:56:57.145286 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757d6301-1a35-4e0b-b87f-795a5f6355a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:56:57.145563 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757d6301-1a35-4e0b-b87f-795a5f6355a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:56:57.153782 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757d6301-1a35-4e0b-b87f-795a5f6355a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:56:57.155736 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757d6301-1a35-4e0b-b87f-795a5f6355a8 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:56:57.164490 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1004/1996] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:56:57 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:56:57.425780 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace0ddb0-34b2-468d-9b07-47f8f219daab service nova] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:56:57.427648 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ace0ddb0-34b2-468d-9b07-47f8f219daab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:56:57.516463 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace0ddb0-34b2-468d-9b07-47f8f219daab service nova] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:56:57.521726 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9bbadb-d19a-42d3-9086-71974e09b72f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:56:57.522432 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9bbadb-d19a-42d3-9086-71974e09b72f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:56:57.523186 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9bbadb-d19a-42d3-9086-71974e09b72f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:56:57.523186 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9bbadb-d19a-42d3-9086-71974e09b72f admin admin] PATH_INFO: `/v3/users/e60638f348384b788807d5aad28d74ad` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:56:57.528509 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9bbadb-d19a-42d3-9086-71974e09b72f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e60638f348384b788807d5aad28d74ad)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:56:57.531068 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9bbadb-d19a-42d3-9086-71974e09b72f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:56:57.599318 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-dd9bbadb-d19a-42d3-9086-71974e09b72f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e60638f348384b788807d5aad28d74ad'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:56:57.613071 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-dd9bbadb-d19a-42d3-9086-71974e09b72f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e60638f348384b788807d5aad28d74ad'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:56:57.614051 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-dd9bbadb-d19a-42d3-9086-71974e09b72f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e60638f348384b788807d5aad28d74ad'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:56:57.629206 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-dd9bbadb-d19a-42d3-9086-71974e09b72f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e60638f348384b788807d5aad28d74ad'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:56:57.636912 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-dd9bbadb-d19a-42d3-9086-71974e09b72f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e60638f348384b788807d5aad28d74ad'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:56:57.643508 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 993/1997] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 03:56:57 2022] DELETE /identity/v3/users/e60638f348384b788807d5aad28d74ad => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:56:57.648458 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44fcfe-20dc-445a-a61e-e3c5a7ecce1e None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:56:57.648752 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44fcfe-20dc-445a-a61e-e3c5a7ecce1e None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:56:57.649079 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a44fcfe-20dc-445a-a61e-e3c5a7ecce1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:56:57.669509 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-0a44fcfe-20dc-445a-a61e-e3c5a7ecce1e None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:56:57.670728 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-0a44fcfe-20dc-445a-a61e-e3c5a7ecce1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:56:57.740246 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1005/1998] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:56:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:56:57.751912 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9bbadb-d19a-42d3-9086-71974e09b72f admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:56:57.753359 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-dd9bbadb-d19a-42d3-9086-71974e09b72f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:56:57.815046 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9bbadb-d19a-42d3-9086-71974e09b72f admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:56:57.818710 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4db4718-b725-4bac-a8d0-12d1135fb970 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:56:57.819323 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db4718-b725-4bac-a8d0-12d1135fb970 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:56:57.819607 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db4718-b725-4bac-a8d0-12d1135fb970 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:56:57.819879 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4db4718-b725-4bac-a8d0-12d1135fb970 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:56:57.829833 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4db4718-b725-4bac-a8d0-12d1135fb970 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:56:57.831282 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4db4718-b725-4bac-a8d0-12d1135fb970 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:56:57.840961 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 994/1999] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:56:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:56:58.119621 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a44fcfe-20dc-445a-a61e-e3c5a7ecce1e None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:56:58.121223 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-0a44fcfe-20dc-445a-a61e-e3c5a7ecce1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:56:58.134397 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a44fcfe-20dc-445a-a61e-e3c5a7ecce1e None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:56:58.139661 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e35b3f6-eb83-4f53-8b0e-b89528f3a888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:56:58.141873 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e35b3f6-eb83-4f53-8b0e-b89528f3a888 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:56:58.141873 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e35b3f6-eb83-4f53-8b0e-b89528f3a888 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:56:58.141873 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e35b3f6-eb83-4f53-8b0e-b89528f3a888 admin admin] PATH_INFO: `/v3/projects/65c936e5fe464fbe873fd325b75df15c` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:56:58.142182 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e35b3f6-eb83-4f53-8b0e-b89528f3a888 admin admin] RBAC: Authorizing `identity:delete_project(project_id=65c936e5fe464fbe873fd325b75df15c)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:56:58.144183 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e35b3f6-eb83-4f53-8b0e-b89528f3a888 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:56:58.194006 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-8e35b3f6-eb83-4f53-8b0e-b89528f3a888 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '65c936e5fe464fbe873fd325b75df15c', 'request_id': 'req-8e35b3f6-eb83-4f53-8b0e-b89528f3a888'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:56:58.200060 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-8e35b3f6-eb83-4f53-8b0e-b89528f3a888 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '65c936e5fe464fbe873fd325b75df15c', 'request_id': 'req-8e35b3f6-eb83-4f53-8b0e-b89528f3a888'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:56:58.207077 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-8e35b3f6-eb83-4f53-8b0e-b89528f3a888 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '65c936e5fe464fbe873fd325b75df15c', 'request_id': 'req-8e35b3f6-eb83-4f53-8b0e-b89528f3a888'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:56:58.219760 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-8e35b3f6-eb83-4f53-8b0e-b89528f3a888 admin admin] The token cache is being invalidate because project 65c936e5fe464fbe873fd325b75df15c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 03:56:58.220089 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-8e35b3f6-eb83-4f53-8b0e-b89528f3a888 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:56:58.220785 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-8e35b3f6-eb83-4f53-8b0e-b89528f3a888 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:56:58.222342 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1006/2000] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 03:56:58 2022] DELETE /identity/v3/projects/65c936e5fe464fbe873fd325b75df15c => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:56:58.226507 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4db4718-b725-4bac-a8d0-12d1135fb970 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:56:58.238949 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4db4718-b725-4bac-a8d0-12d1135fb970 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:56:58.258482 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:56:58.300018 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c45383a-664c-429f-9680-9b718d2222b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:56:58.300664 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c45383a-664c-429f-9680-9b718d2222b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:56:58.300931 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c45383a-664c-429f-9680-9b718d2222b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:56:58.301197 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c45383a-664c-429f-9680-9b718d2222b8 admin admin] PATH_INFO: `/v3/users/5b08a5b625d4487683f211c37dfaa88e` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:56:58.302363 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c45383a-664c-429f-9680-9b718d2222b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b08a5b625d4487683f211c37dfaa88e)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:56:58.304388 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c45383a-664c-429f-9680-9b718d2222b8 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:56:58.351655 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8c45383a-664c-429f-9680-9b718d2222b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b08a5b625d4487683f211c37dfaa88e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:56:58.358975 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8c45383a-664c-429f-9680-9b718d2222b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b08a5b625d4487683f211c37dfaa88e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:56:58.359640 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8c45383a-664c-429f-9680-9b718d2222b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b08a5b625d4487683f211c37dfaa88e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:56:58.371431 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8c45383a-664c-429f-9680-9b718d2222b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b08a5b625d4487683f211c37dfaa88e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:56:58.379866 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8c45383a-664c-429f-9680-9b718d2222b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b08a5b625d4487683f211c37dfaa88e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:56:58.385802 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 995/2001] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 03:56:58 2022] DELETE /identity/v3/users/5b08a5b625d4487683f211c37dfaa88e => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:56:58.661077 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e35b3f6-eb83-4f53-8b0e-b89528f3a888 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:56:58.662730 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-8e35b3f6-eb83-4f53-8b0e-b89528f3a888 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:56:58.676802 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e35b3f6-eb83-4f53-8b0e-b89528f3a888 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:56:58.689204 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:56:58.736534 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa23b8de-5ca3-48f9-b52f-906eda64aca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:56:58.737170 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa23b8de-5ca3-48f9-b52f-906eda64aca8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:56:58.737446 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa23b8de-5ca3-48f9-b52f-906eda64aca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:56:58.737715 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa23b8de-5ca3-48f9-b52f-906eda64aca8 admin admin] PATH_INFO: `/v3/projects/5286b614774c4eb48550cb53f67707eb` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:56:58.738822 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa23b8de-5ca3-48f9-b52f-906eda64aca8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5286b614774c4eb48550cb53f67707eb)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:56:58.740755 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa23b8de-5ca3-48f9-b52f-906eda64aca8 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:56:58.786634 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-fa23b8de-5ca3-48f9-b52f-906eda64aca8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5286b614774c4eb48550cb53f67707eb', 'request_id': 'req-fa23b8de-5ca3-48f9-b52f-906eda64aca8'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:56:58.792396 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-fa23b8de-5ca3-48f9-b52f-906eda64aca8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5286b614774c4eb48550cb53f67707eb', 'request_id': 'req-fa23b8de-5ca3-48f9-b52f-906eda64aca8'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:56:58.798981 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-fa23b8de-5ca3-48f9-b52f-906eda64aca8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5286b614774c4eb48550cb53f67707eb', 'request_id': 'req-fa23b8de-5ca3-48f9-b52f-906eda64aca8'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:56:58.810756 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-fa23b8de-5ca3-48f9-b52f-906eda64aca8 admin admin] The token cache is being invalidate because project 5286b614774c4eb48550cb53f67707eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 03:56:58.811041 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-fa23b8de-5ca3-48f9-b52f-906eda64aca8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:56:58.811700 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-fa23b8de-5ca3-48f9-b52f-906eda64aca8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:56:58.813218 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1007/2002] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 03:56:58 2022] DELETE /identity/v3/projects/5286b614774c4eb48550cb53f67707eb => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 03:56:58.818176 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c79d260-2b30-4431-8965-bba38d40b44d None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:56:58.818695 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c79d260-2b30-4431-8965-bba38d40b44d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:56:58.818809 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c79d260-2b30-4431-8965-bba38d40b44d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:56:58.837980 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-9c79d260-2b30-4431-8965-bba38d40b44d None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:56:58.839180 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9c79d260-2b30-4431-8965-bba38d40b44d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:56:58.898290 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 996/2003] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:56:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:56:58.906587 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab8659-b219-43da-82e0-8309edb9757b None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:56:58.906875 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab8659-b219-43da-82e0-8309edb9757b None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:56:58.907151 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab8659-b219-43da-82e0-8309edb9757b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:56:58.927126 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-d7ab8659-b219-43da-82e0-8309edb9757b None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:56:58.928369 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d7ab8659-b219-43da-82e0-8309edb9757b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:56:58.936484 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1008/2004] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:56:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:56:58.940522 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c79d260-2b30-4431-8965-bba38d40b44d None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:56:58.950277 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c79d260-2b30-4431-8965-bba38d40b44d None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:56:58.954837 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4d2557-7620-490c-b3ab-fd3354a77a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:56:58.955455 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4d2557-7620-490c-b3ab-fd3354a77a25 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:56:58.955734 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4d2557-7620-490c-b3ab-fd3354a77a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:56:58.956044 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4d2557-7620-490c-b3ab-fd3354a77a25 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:56:58.956694 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4d2557-7620-490c-b3ab-fd3354a77a25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:56:58.956996 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4d2557-7620-490c-b3ab-fd3354a77a25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:56:58.958600 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4d2557-7620-490c-b3ab-fd3354a77a25 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:56:58.971388 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 997/2005] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 03:56:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:56:58.975656 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab8659-b219-43da-82e0-8309edb9757b None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:56:58.985729 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab8659-b219-43da-82e0-8309edb9757b None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:56:58.990511 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b7428a-cc48-4729-b695-1641e43a2a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:56:58.991127 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b7428a-cc48-4729-b695-1641e43a2a27 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:56:58.991411 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b7428a-cc48-4729-b695-1641e43a2a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:56:58.991682 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b7428a-cc48-4729-b695-1641e43a2a27 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:56:58.992384 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b7428a-cc48-4729-b695-1641e43a2a27 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:56:58.994287 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b7428a-cc48-4729-b695-1641e43a2a27 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:56:59.025686 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1009/2006] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:56:58 2022] POST /identity/v3/projects => generated 365 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:56:59.030391 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4d2557-7620-490c-b3ab-fd3354a77a25 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:56:59.040623 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4d2557-7620-490c-b3ab-fd3354a77a25 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:56:59.066614 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:56:59.100147 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8a17e1-1fc9-46e6-b208-267b247cf09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:56:59.100753 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a17e1-1fc9-46e6-b208-267b247cf09b admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:56:59.101034 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a17e1-1fc9-46e6-b208-267b247cf09b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:56:59.101307 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8a17e1-1fc9-46e6-b208-267b247cf09b admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:56:59.101931 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8a17e1-1fc9-46e6-b208-267b247cf09b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:56:59.103845 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8a17e1-1fc9-46e6-b208-267b247cf09b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:56:59.123668 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 998/2007] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:56:59 2022] POST /identity/v3/users => generated 283 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:56:59.128143 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b7428a-cc48-4729-b695-1641e43a2a27 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:56:59.139130 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b7428a-cc48-4729-b695-1641e43a2a27 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:56:59.143925 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98575a93-7790-40fa-a18a-e128ba10590e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:56:59.144559 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98575a93-7790-40fa-a18a-e128ba10590e admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:56:59.144856 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98575a93-7790-40fa-a18a-e128ba10590e admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:56:59.145123 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98575a93-7790-40fa-a18a-e128ba10590e admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:56:59.145778 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98575a93-7790-40fa-a18a-e128ba10590e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:56:59.147382 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98575a93-7790-40fa-a18a-e128ba10590e admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:56:59.158043 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1010/2008] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:56:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:56:59.162180 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8a17e1-1fc9-46e6-b208-267b247cf09b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:56:59.172064 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8a17e1-1fc9-46e6-b208-267b247cf09b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:56:59.177181 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc97d938-7731-456f-9dc6-57a6a09b3e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:56:59.177962 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc97d938-7731-456f-9dc6-57a6a09b3e17 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:56:59.178261 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc97d938-7731-456f-9dc6-57a6a09b3e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:56:59.178653 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc97d938-7731-456f-9dc6-57a6a09b3e17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:56:59.179343 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc97d938-7731-456f-9dc6-57a6a09b3e17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:56:59.181227 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc97d938-7731-456f-9dc6-57a6a09b3e17 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:56:59.193728 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 999/2009] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:56:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:56:59.198191 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98575a93-7790-40fa-a18a-e128ba10590e admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:56:59.208945 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98575a93-7790-40fa-a18a-e128ba10590e admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:56:59.213603 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5f9e6b-fdab-4e28-86d0-d33b7feafd57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:56:59.214252 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f9e6b-fdab-4e28-86d0-d33b7feafd57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:56:59.214533 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f9e6b-fdab-4e28-86d0-d33b7feafd57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:56:59.214807 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5f9e6b-fdab-4e28-86d0-d33b7feafd57 admin admin] PATH_INFO: `/v3/projects/f59346ba354d4890ad7ad734d6dc720f/users/19e10d37352641959e12f0ca2f57bca3/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:56:59.229846 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5f9e6b-fdab-4e28-86d0-d33b7feafd57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f59346ba354d4890ad7ad734d6dc720f, user_id=19e10d37352641959e12f0ca2f57bca3, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:56:59.232536 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5f9e6b-fdab-4e28-86d0-d33b7feafd57 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:56:59.244782 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1011/2010] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:56:59 2022] PUT /identity/v3/projects/f59346ba354d4890ad7ad734d6dc720f/users/19e10d37352641959e12f0ca2f57bca3/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:56:59.311448 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b3f53b-8354-4d25-8aa6-4921ba09ec79 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:56:59.311746 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b3f53b-8354-4d25-8aa6-4921ba09ec79 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:56:59.312044 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b3f53b-8354-4d25-8aa6-4921ba09ec79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:56:59.339747 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d9b3f53b-8354-4d25-8aa6-4921ba09ec79 None None] MFA Rules not processed for user `19e10d37352641959e12f0ca2f57bca3`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:56:59.340994 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d9b3f53b-8354-4d25-8aa6-4921ba09ec79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:56:59.405168 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1000/2011] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:56:59 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:56:59.409699 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5f9e6b-fdab-4e28-86d0-d33b7feafd57 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:56:59.454035 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:56:59.472559 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5f9e6b-fdab-4e28-86d0-d33b7feafd57 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:56:59.477293 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d71de1-ac46-4224-a1ad-85a9cb872193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:56:59.477899 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d71de1-ac46-4224-a1ad-85a9cb872193 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:56:59.478181 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d71de1-ac46-4224-a1ad-85a9cb872193 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:56:59.478480 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d71de1-ac46-4224-a1ad-85a9cb872193 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:56:59.479123 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d71de1-ac46-4224-a1ad-85a9cb872193 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:56:59.481068 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d71de1-ac46-4224-a1ad-85a9cb872193 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:56:59.507336 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1012/2012] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:56:59 2022] POST /identity/v3/projects => generated 363 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:56:59.511554 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b3f53b-8354-4d25-8aa6-4921ba09ec79 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:56:59.522844 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b3f53b-8354-4d25-8aa6-4921ba09ec79 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:56:59.579292 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295bbb4e-0be5-4f44-a4c9-2ef809de5e66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:56:59.579907 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295bbb4e-0be5-4f44-a4c9-2ef809de5e66 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:56:59.580218 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295bbb4e-0be5-4f44-a4c9-2ef809de5e66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:56:59.580491 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295bbb4e-0be5-4f44-a4c9-2ef809de5e66 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:56:59.581143 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295bbb4e-0be5-4f44-a4c9-2ef809de5e66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:56:59.583083 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295bbb4e-0be5-4f44-a4c9-2ef809de5e66 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:56:59.607046 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1001/2013] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:56:59 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:56:59.608213 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d71de1-ac46-4224-a1ad-85a9cb872193 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:56:59.618329 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d71de1-ac46-4224-a1ad-85a9cb872193 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:56:59.623644 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835b3c37-3354-46db-bb35-0ca2ad0485ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:56:59.624308 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b3c37-3354-46db-bb35-0ca2ad0485ec admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:56:59.624847 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b3c37-3354-46db-bb35-0ca2ad0485ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:56:59.625127 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b3c37-3354-46db-bb35-0ca2ad0485ec admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:56:59.625770 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835b3c37-3354-46db-bb35-0ca2ad0485ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:56:59.627431 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835b3c37-3354-46db-bb35-0ca2ad0485ec admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:56:59.638124 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1013/2014] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:56:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 03:56:59.642624 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295bbb4e-0be5-4f44-a4c9-2ef809de5e66 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:56:59.652535 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295bbb4e-0be5-4f44-a4c9-2ef809de5e66 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:56:59.657198 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e646fd87-29ca-4d59-b2de-b09806b92993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:56:59.657803 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646fd87-29ca-4d59-b2de-b09806b92993 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:56:59.658083 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646fd87-29ca-4d59-b2de-b09806b92993 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:56:59.658359 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646fd87-29ca-4d59-b2de-b09806b92993 admin admin] PATH_INFO: `/v3/projects/312fc69cc2f34fd299b2ca7cfcd00196/users/ac3c9f7dfe0f47c989cad25b8e524d87/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:56:59.674269 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e646fd87-29ca-4d59-b2de-b09806b92993 admin admin] RBAC: Authorizing `identity:create_grant(project_id=312fc69cc2f34fd299b2ca7cfcd00196, user_id=ac3c9f7dfe0f47c989cad25b8e524d87, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:56:59.676971 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e646fd87-29ca-4d59-b2de-b09806b92993 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:56:59.690274 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1002/2015] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:56:59 2022] PUT /identity/v3/projects/312fc69cc2f34fd299b2ca7cfcd00196/users/ac3c9f7dfe0f47c989cad25b8e524d87/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:56:59.694215 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835b3c37-3354-46db-bb35-0ca2ad0485ec admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:56:59.704710 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835b3c37-3354-46db-bb35-0ca2ad0485ec admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:56:59.760579 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f3d603-ca25-4d6b-b164-541070bee11b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:56:59.761186 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f3d603-ca25-4d6b-b164-541070bee11b admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:56:59.761465 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f3d603-ca25-4d6b-b164-541070bee11b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:56:59.761786 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f3d603-ca25-4d6b-b164-541070bee11b admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:56:59.762425 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f3d603-ca25-4d6b-b164-541070bee11b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:56:59.764086 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f3d603-ca25-4d6b-b164-541070bee11b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:56:59.773751 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1014/2016] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:56:59 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:56:59.778007 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e646fd87-29ca-4d59-b2de-b09806b92993 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:56:59.783367 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:56:59.789622 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e646fd87-29ca-4d59-b2de-b09806b92993 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:56:59.795215 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa377cd-ab5d-41a1-af0b-a903f31b430c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:56:59.795904 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa377cd-ab5d-41a1-af0b-a903f31b430c admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:56:59.796215 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa377cd-ab5d-41a1-af0b-a903f31b430c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:56:59.796489 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa377cd-ab5d-41a1-af0b-a903f31b430c admin admin] PATH_INFO: `/v3/projects/312fc69cc2f34fd299b2ca7cfcd00196/users/ac3c9f7dfe0f47c989cad25b8e524d87/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:56:59.798393 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa377cd-ab5d-41a1-af0b-a903f31b430c admin admin] RBAC: Authorizing `identity:create_grant(project_id=312fc69cc2f34fd299b2ca7cfcd00196, user_id=ac3c9f7dfe0f47c989cad25b8e524d87, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:56:59.801149 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa377cd-ab5d-41a1-af0b-a903f31b430c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:56:59.811840 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1003/2017] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:56:59 2022] PUT /identity/v3/projects/312fc69cc2f34fd299b2ca7cfcd00196/users/ac3c9f7dfe0f47c989cad25b8e524d87/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:56:59.821505 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d384f77-e318-4b22-80e3-e3259fcff6dd None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:56:59.821774 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d384f77-e318-4b22-80e3-e3259fcff6dd None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:56:59.822050 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d384f77-e318-4b22-80e3-e3259fcff6dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:56:59.849104 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-7d384f77-e318-4b22-80e3-e3259fcff6dd None None] MFA Rules not processed for user `ac3c9f7dfe0f47c989cad25b8e524d87`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:56:59.850307 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-7d384f77-e318-4b22-80e3-e3259fcff6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:56:59.918088 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1015/2018] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:56:59 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 03:57:09.807570 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=4,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:57:09.915095 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=82,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:57:37.325292 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa377cd-ab5d-41a1-af0b-a903f31b430c admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:57:37.327640 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1aa377cd-ab5d-41a1-af0b-a903f31b430c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:57:37.389637 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa377cd-ab5d-41a1-af0b-a903f31b430c admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:57:37.393378 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf143b9-ad9e-4531-8658-2a0bd87e14a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:57:37.393987 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf143b9-ad9e-4531-8658-2a0bd87e14a3 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:57:37.394268 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf143b9-ad9e-4531-8658-2a0bd87e14a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:57:37.394541 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf143b9-ad9e-4531-8658-2a0bd87e14a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:57:37.410915 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf143b9-ad9e-4531-8658-2a0bd87e14a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:57:37.412691 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf143b9-ad9e-4531-8658-2a0bd87e14a3 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:57:37.422480 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1004/2019] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:57:37 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:57:41.235721 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d384f77-e318-4b22-80e3-e3259fcff6dd None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:57:41.237148 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-7d384f77-e318-4b22-80e3-e3259fcff6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:57:41.293434 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d384f77-e318-4b22-80e3-e3259fcff6dd None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:57:41.297027 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deaa59e-d943-44ce-8b89-120b09f2c0f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:57:41.297629 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deaa59e-d943-44ce-8b89-120b09f2c0f9 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:57:41.297906 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deaa59e-d943-44ce-8b89-120b09f2c0f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:57:41.298178 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deaa59e-d943-44ce-8b89-120b09f2c0f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:57:41.299896 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-8deaa59e-d943-44ce-8b89-120b09f2c0f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:57:41.368253 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deaa59e-d943-44ce-8b89-120b09f2c0f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:57:41.369677 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8deaa59e-d943-44ce-8b89-120b09f2c0f9 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:57:41.381181 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1016/2020] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:57:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:57:44.975902 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf143b9-ad9e-4531-8658-2a0bd87e14a3 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:57:44.977350 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8bf143b9-ad9e-4531-8658-2a0bd87e14a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:57:45.049399 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf143b9-ad9e-4531-8658-2a0bd87e14a3 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:57:45.054410 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b75a91-b248-414d-9b7c-6691e9a18c8b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4b9bd741574ddabadd4b7651ebc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:57:45.055028 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b75a91-b248-414d-9b7c-6691e9a18c8b service placement] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:57:45.055304 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b75a91-b248-414d-9b7c-6691e9a18c8b service placement] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:57:45.055575 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b75a91-b248-414d-9b7c-6691e9a18c8b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:57:45.057350 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a1b75a91-b248-414d-9b7c-6691e9a18c8b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:57:45.069326 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b75a91-b248-414d-9b7c-6691e9a18c8b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:57:45.070894 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b75a91-b248-414d-9b7c-6691e9a18c8b service placement] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:57:45.080651 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1005/2021] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:57:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:57:51.376405 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=54 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:57:55.076757 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=58 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:58:57.733508 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deaa59e-d943-44ce-8b89-120b09f2c0f9 service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:58:57.734980 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-8deaa59e-d943-44ce-8b89-120b09f2c0f9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:58:57.747675 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deaa59e-d943-44ce-8b89-120b09f2c0f9 service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:58:57.751756 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93de0672-4be6-47f9-9cb7-3378c359be63 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:58:57.752384 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de0672-4be6-47f9-9cb7-3378c359be63 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:58:57.752668 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de0672-4be6-47f9-9cb7-3378c359be63 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:58:57.752951 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93de0672-4be6-47f9-9cb7-3378c359be63 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:58:57.754743 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-93de0672-4be6-47f9-9cb7-3378c359be63 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:58:57.815752 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93de0672-4be6-47f9-9cb7-3378c359be63 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:58:57.817144 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93de0672-4be6-47f9-9cb7-3378c359be63 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:58:57.826190 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1017/2022] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:58:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:58:58.090448 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b75a91-b248-414d-9b7c-6691e9a18c8b service placement] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:58:58.153852 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b75a91-b248-414d-9b7c-6691e9a18c8b service placement] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:58:58.158732 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53551f68-e019-4f8b-ae0b-ee99d286e2ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:58:58.159905 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53551f68-e019-4f8b-ae0b-ee99d286e2ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:58:58.160217 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53551f68-e019-4f8b-ae0b-ee99d286e2ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:58:58.160497 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53551f68-e019-4f8b-ae0b-ee99d286e2ab admin admin] PATH_INFO: `/v3/users/19e10d37352641959e12f0ca2f57bca3` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:58:58.161630 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53551f68-e019-4f8b-ae0b-ee99d286e2ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=19e10d37352641959e12f0ca2f57bca3)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:58:58.163547 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53551f68-e019-4f8b-ae0b-ee99d286e2ab admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:58:58.215778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-53551f68-e019-4f8b-ae0b-ee99d286e2ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19e10d37352641959e12f0ca2f57bca3'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:58:58.223837 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-53551f68-e019-4f8b-ae0b-ee99d286e2ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19e10d37352641959e12f0ca2f57bca3'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:58:58.224621 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-53551f68-e019-4f8b-ae0b-ee99d286e2ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19e10d37352641959e12f0ca2f57bca3'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:58:58.237604 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-53551f68-e019-4f8b-ae0b-ee99d286e2ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19e10d37352641959e12f0ca2f57bca3'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:58:58.244762 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-53551f68-e019-4f8b-ae0b-ee99d286e2ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19e10d37352641959e12f0ca2f57bca3'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:58:58.251552 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1006/2023] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 03:58:58 2022] DELETE /identity/v3/users/19e10d37352641959e12f0ca2f57bca3 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:58:58.255788 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c645ce-63be-4240-976f-7dee9fe6c03d None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:58:58.256172 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c645ce-63be-4240-976f-7dee9fe6c03d None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:58:58.256450 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c645ce-63be-4240-976f-7dee9fe6c03d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:58:58.276953 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-59c645ce-63be-4240-976f-7dee9fe6c03d None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:58:58.278707 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-59c645ce-63be-4240-976f-7dee9fe6c03d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:58:58.356384 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1018/2024] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:58:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:58:58.367718 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53551f68-e019-4f8b-ae0b-ee99d286e2ab admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:58:58.369010 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-53551f68-e019-4f8b-ae0b-ee99d286e2ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:58:58.437994 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53551f68-e019-4f8b-ae0b-ee99d286e2ab admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:58:58.442130 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83263a31-8c99-4fcd-b36e-b31ae0bede2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:58:58.442797 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83263a31-8c99-4fcd-b36e-b31ae0bede2b service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:58:58.443188 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83263a31-8c99-4fcd-b36e-b31ae0bede2b service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:58:58.443437 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83263a31-8c99-4fcd-b36e-b31ae0bede2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:58:58.453138 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83263a31-8c99-4fcd-b36e-b31ae0bede2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:58:58.454571 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83263a31-8c99-4fcd-b36e-b31ae0bede2b service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:58:58.464252 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1007/2025] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:58:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:58:58.741461 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c645ce-63be-4240-976f-7dee9fe6c03d None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:58:58.742905 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-59c645ce-63be-4240-976f-7dee9fe6c03d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:58:58.756216 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c645ce-63be-4240-976f-7dee9fe6c03d None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:58:58.767588 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a77b861-d5ca-4e16-8a9f-569c800f00c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:58:58.768473 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a77b861-d5ca-4e16-8a9f-569c800f00c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:58:58.768853 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a77b861-d5ca-4e16-8a9f-569c800f00c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:58:58.769212 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a77b861-d5ca-4e16-8a9f-569c800f00c3 admin admin] PATH_INFO: `/v3/projects/f59346ba354d4890ad7ad734d6dc720f` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:58:58.770631 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a77b861-d5ca-4e16-8a9f-569c800f00c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f59346ba354d4890ad7ad734d6dc720f)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:58:58.773493 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a77b861-d5ca-4e16-8a9f-569c800f00c3 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:58:58.822598 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1a77b861-d5ca-4e16-8a9f-569c800f00c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f59346ba354d4890ad7ad734d6dc720f', 'request_id': 'req-1a77b861-d5ca-4e16-8a9f-569c800f00c3'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:58:58.829100 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1a77b861-d5ca-4e16-8a9f-569c800f00c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f59346ba354d4890ad7ad734d6dc720f', 'request_id': 'req-1a77b861-d5ca-4e16-8a9f-569c800f00c3'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:58:58.835443 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1a77b861-d5ca-4e16-8a9f-569c800f00c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f59346ba354d4890ad7ad734d6dc720f', 'request_id': 'req-1a77b861-d5ca-4e16-8a9f-569c800f00c3'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:58:58.849596 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1a77b861-d5ca-4e16-8a9f-569c800f00c3 admin admin] The token cache is being invalidate because project f59346ba354d4890ad7ad734d6dc720f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 03:58:58.849888 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1a77b861-d5ca-4e16-8a9f-569c800f00c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:58:58.850509 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1a77b861-d5ca-4e16-8a9f-569c800f00c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:58:58.852177 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1019/2026] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 03:58:58 2022] DELETE /identity/v3/projects/f59346ba354d4890ad7ad734d6dc720f => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:58:58.859740 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83263a31-8c99-4fcd-b36e-b31ae0bede2b service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:58:58.869505 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83263a31-8c99-4fcd-b36e-b31ae0bede2b service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:58:58.888762 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:58:58.927196 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88216f6e-a06f-43f4-b63f-0cf184ba8204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:58:58.927823 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88216f6e-a06f-43f4-b63f-0cf184ba8204 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:58:58.928138 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88216f6e-a06f-43f4-b63f-0cf184ba8204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:58:58.928413 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88216f6e-a06f-43f4-b63f-0cf184ba8204 admin admin] PATH_INFO: `/v3/users/ac3c9f7dfe0f47c989cad25b8e524d87` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:58:58.929552 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88216f6e-a06f-43f4-b63f-0cf184ba8204 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac3c9f7dfe0f47c989cad25b8e524d87)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:58:58.931467 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88216f6e-a06f-43f4-b63f-0cf184ba8204 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:58:58.976254 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-88216f6e-a06f-43f4-b63f-0cf184ba8204 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac3c9f7dfe0f47c989cad25b8e524d87'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:58:58.983563 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-88216f6e-a06f-43f4-b63f-0cf184ba8204 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac3c9f7dfe0f47c989cad25b8e524d87'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:58:58.984217 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-88216f6e-a06f-43f4-b63f-0cf184ba8204 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac3c9f7dfe0f47c989cad25b8e524d87'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:58:58.997206 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-88216f6e-a06f-43f4-b63f-0cf184ba8204 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac3c9f7dfe0f47c989cad25b8e524d87'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:58:59.004548 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-88216f6e-a06f-43f4-b63f-0cf184ba8204 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac3c9f7dfe0f47c989cad25b8e524d87'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:58:59.010354 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1008/2027] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 03:58:58 2022] DELETE /identity/v3/users/ac3c9f7dfe0f47c989cad25b8e524d87 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 03:58:59.287985 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a77b861-d5ca-4e16-8a9f-569c800f00c3 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:58:59.289538 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-1a77b861-d5ca-4e16-8a9f-569c800f00c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:58:59.307291 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a77b861-d5ca-4e16-8a9f-569c800f00c3 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:58:59.331115 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:58:59.372759 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad82aa74-6ca6-453d-a424-2f72d95b0297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:58:59.373307 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad82aa74-6ca6-453d-a424-2f72d95b0297 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:58:59.373582 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad82aa74-6ca6-453d-a424-2f72d95b0297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:58:59.373857 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad82aa74-6ca6-453d-a424-2f72d95b0297 admin admin] PATH_INFO: `/v3/projects/312fc69cc2f34fd299b2ca7cfcd00196` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:58:59.375228 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad82aa74-6ca6-453d-a424-2f72d95b0297 admin admin] RBAC: Authorizing `identity:delete_project(project_id=312fc69cc2f34fd299b2ca7cfcd00196)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:58:59.377155 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad82aa74-6ca6-453d-a424-2f72d95b0297 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:58:59.425953 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ad82aa74-6ca6-453d-a424-2f72d95b0297 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '312fc69cc2f34fd299b2ca7cfcd00196', 'request_id': 'req-ad82aa74-6ca6-453d-a424-2f72d95b0297'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:58:59.431917 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ad82aa74-6ca6-453d-a424-2f72d95b0297 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '312fc69cc2f34fd299b2ca7cfcd00196', 'request_id': 'req-ad82aa74-6ca6-453d-a424-2f72d95b0297'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:58:59.438957 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ad82aa74-6ca6-453d-a424-2f72d95b0297 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '312fc69cc2f34fd299b2ca7cfcd00196', 'request_id': 'req-ad82aa74-6ca6-453d-a424-2f72d95b0297'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:58:59.451550 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ad82aa74-6ca6-453d-a424-2f72d95b0297 admin admin] The token cache is being invalidate because project 312fc69cc2f34fd299b2ca7cfcd00196 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 03:58:59.451929 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ad82aa74-6ca6-453d-a424-2f72d95b0297 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:58:59.452585 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ad82aa74-6ca6-453d-a424-2f72d95b0297 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 03:58:59.454078 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1020/2028] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 03:58:59 2022] DELETE /identity/v3/projects/312fc69cc2f34fd299b2ca7cfcd00196 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 03:58:59.458938 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d0d52a-9880-4025-984e-0d18cfefce0b None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:58:59.459440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d0d52a-9880-4025-984e-0d18cfefce0b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:58:59.459566 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d0d52a-9880-4025-984e-0d18cfefce0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:58:59.479879 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-e8d0d52a-9880-4025-984e-0d18cfefce0b None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:58:59.481655 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e8d0d52a-9880-4025-984e-0d18cfefce0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:58:59.545023 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1009/2029] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:58:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:58:59.552734 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f9ab3c-e16d-46ca-a6c5-e74474206447 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:58:59.553021 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f9ab3c-e16d-46ca-a6c5-e74474206447 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:58:59.553320 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f9ab3c-e16d-46ca-a6c5-e74474206447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:58:59.573448 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-a0f9ab3c-e16d-46ca-a6c5-e74474206447 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:58:59.574679 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a0f9ab3c-e16d-46ca-a6c5-e74474206447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:58:59.582617 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1021/2030] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:58:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:58:59.586812 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d0d52a-9880-4025-984e-0d18cfefce0b None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:58:59.598431 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d0d52a-9880-4025-984e-0d18cfefce0b None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:58:59.604479 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9441adb0-69c9-401f-b090-8608aa8ba2ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:58:59.606072 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9441adb0-69c9-401f-b090-8608aa8ba2ad admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:58:59.607119 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9441adb0-69c9-401f-b090-8608aa8ba2ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:58:59.607512 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9441adb0-69c9-401f-b090-8608aa8ba2ad admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:58:59.608356 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9441adb0-69c9-401f-b090-8608aa8ba2ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 03:58:59.608760 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9441adb0-69c9-401f-b090-8608aa8ba2ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:58:59.610621 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9441adb0-69c9-401f-b090-8608aa8ba2ad admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:58:59.627434 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1010/2031] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 03:58:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 03:58:59.628537 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f9ab3c-e16d-46ca-a6c5-e74474206447 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:58:59.638085 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f9ab3c-e16d-46ca-a6c5-e74474206447 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:58:59.642892 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc80d321-736c-48de-92f1-3d308fed7539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:58:59.643496 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80d321-736c-48de-92f1-3d308fed7539 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:58:59.643775 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80d321-736c-48de-92f1-3d308fed7539 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:58:59.644079 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc80d321-736c-48de-92f1-3d308fed7539 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:58:59.644714 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc80d321-736c-48de-92f1-3d308fed7539 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:58:59.646603 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc80d321-736c-48de-92f1-3d308fed7539 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:58:59.684790 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1022/2032] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:58:59 2022] POST /identity/v3/projects => generated 401 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:58:59.689492 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9441adb0-69c9-401f-b090-8608aa8ba2ad admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:58:59.700541 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9441adb0-69c9-401f-b090-8608aa8ba2ad admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:58:59.727823 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:58:59.760822 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ed0d74-c88d-4168-b210-48696cb76b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:58:59.761960 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ed0d74-c88d-4168-b210-48696cb76b49 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:58:59.762237 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ed0d74-c88d-4168-b210-48696cb76b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:58:59.762542 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ed0d74-c88d-4168-b210-48696cb76b49 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:58:59.763174 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ed0d74-c88d-4168-b210-48696cb76b49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:58:59.765130 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ed0d74-c88d-4168-b210-48696cb76b49 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:58:59.790427 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1011/2033] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:58:59 2022] POST /identity/v3/users => generated 301 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 03:58:59.791523 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc80d321-736c-48de-92f1-3d308fed7539 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:58:59.801959 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc80d321-736c-48de-92f1-3d308fed7539 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:58:59.806628 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9de8a50-1c7d-4a1e-b919-c8fcee18718e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:58:59.807230 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de8a50-1c7d-4a1e-b919-c8fcee18718e admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:58:59.807513 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de8a50-1c7d-4a1e-b919-c8fcee18718e admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:58:59.807792 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9de8a50-1c7d-4a1e-b919-c8fcee18718e admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:58:59.808502 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9de8a50-1c7d-4a1e-b919-c8fcee18718e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:58:59.810131 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9de8a50-1c7d-4a1e-b919-c8fcee18718e admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:58:59.825221 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1023/2034] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:58:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:58:59.825344 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ed0d74-c88d-4168-b210-48696cb76b49 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:58:59.841783 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ed0d74-c88d-4168-b210-48696cb76b49 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:58:59.846749 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd98b26-b696-4b13-89f2-e6441e21cb21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:58:59.847371 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd98b26-b696-4b13-89f2-e6441e21cb21 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:58:59.847648 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd98b26-b696-4b13-89f2-e6441e21cb21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:58:59.847925 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd98b26-b696-4b13-89f2-e6441e21cb21 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:58:59.848601 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd98b26-b696-4b13-89f2-e6441e21cb21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:58:59.850202 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd98b26-b696-4b13-89f2-e6441e21cb21 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:58:59.866042 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1012/2035] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:58:59 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:58:59.867130 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9de8a50-1c7d-4a1e-b919-c8fcee18718e admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:58:59.886872 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9de8a50-1c7d-4a1e-b919-c8fcee18718e admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:58:59.893316 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beac0d4d-04c9-4672-9550-18ab2dea941c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:58:59.894253 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beac0d4d-04c9-4672-9550-18ab2dea941c admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:58:59.895102 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beac0d4d-04c9-4672-9550-18ab2dea941c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:58:59.895738 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beac0d4d-04c9-4672-9550-18ab2dea941c admin admin] PATH_INFO: `/v3/projects/264dfc1ce7e242929e8bcd650c248cd3/users/0325cee0036f490c9f732d73a58d128a/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:58:59.921596 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beac0d4d-04c9-4672-9550-18ab2dea941c admin admin] RBAC: Authorizing `identity:create_grant(project_id=264dfc1ce7e242929e8bcd650c248cd3, user_id=0325cee0036f490c9f732d73a58d128a, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:58:59.924580 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beac0d4d-04c9-4672-9550-18ab2dea941c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:58:59.967452 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1024/2036] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:58:59 2022] PUT /identity/v3/projects/264dfc1ce7e242929e8bcd650c248cd3/users/0325cee0036f490c9f732d73a58d128a/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 03:58:59.979675 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d304a21-2482-43b2-b26c-e4aceced0a6e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:58:59.980029 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d304a21-2482-43b2-b26c-e4aceced0a6e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:58:59.980335 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d304a21-2482-43b2-b26c-e4aceced0a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:59:00.020465 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-0d304a21-2482-43b2-b26c-e4aceced0a6e None None] MFA Rules not processed for user `0325cee0036f490c9f732d73a58d128a`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:59:00.021819 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0d304a21-2482-43b2-b26c-e4aceced0a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:59:00.104131 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1013/2037] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:58:59 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:59:00.104773 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beac0d4d-04c9-4672-9550-18ab2dea941c admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:59:00.148263 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:59:00.169843 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beac0d4d-04c9-4672-9550-18ab2dea941c admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:59:00.174746 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedada2e-991c-4940-955f-14b2ccfbfaaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:59:00.175429 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedada2e-991c-4940-955f-14b2ccfbfaaa admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:59:00.175782 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedada2e-991c-4940-955f-14b2ccfbfaaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:59:00.176154 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dedada2e-991c-4940-955f-14b2ccfbfaaa admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:59:00.176816 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedada2e-991c-4940-955f-14b2ccfbfaaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:59:00.178725 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dedada2e-991c-4940-955f-14b2ccfbfaaa admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:59:00.209993 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1025/2038] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 03:59:00 2022] POST /identity/v3/projects => generated 401 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 03:59:00.211163 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d304a21-2482-43b2-b26c-e4aceced0a6e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:59:00.222894 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d304a21-2482-43b2-b26c-e4aceced0a6e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:59:00.280444 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8966e0e-b635-4379-8832-fc6c8d7f3d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:59:00.281086 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8966e0e-b635-4379-8832-fc6c8d7f3d99 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:59:00.281364 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8966e0e-b635-4379-8832-fc6c8d7f3d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:59:00.281636 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8966e0e-b635-4379-8832-fc6c8d7f3d99 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:59:00.282267 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8966e0e-b635-4379-8832-fc6c8d7f3d99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:59:00.284195 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8966e0e-b635-4379-8832-fc6c8d7f3d99 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:59:00.309949 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1014/2039] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 03:59:00 2022] POST /identity/v3/users => generated 307 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 03:59:00.311247 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedada2e-991c-4940-955f-14b2ccfbfaaa admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:59:00.326246 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dedada2e-991c-4940-955f-14b2ccfbfaaa admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:59:00.331104 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7c52de-1e05-463b-8871-d173716ac76e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:59:00.331717 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7c52de-1e05-463b-8871-d173716ac76e admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:59:00.332025 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7c52de-1e05-463b-8871-d173716ac76e admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:59:00.332302 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7c52de-1e05-463b-8871-d173716ac76e admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:59:00.332995 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7c52de-1e05-463b-8871-d173716ac76e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:59:00.334600 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7c52de-1e05-463b-8871-d173716ac76e admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:59:00.348531 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1026/2040] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:59:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:59:00.349687 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8966e0e-b635-4379-8832-fc6c8d7f3d99 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:59:00.359549 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8966e0e-b635-4379-8832-fc6c8d7f3d99 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:59:00.364187 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67031e57-94b5-48c3-96f3-c3ba6ff38668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:59:00.364853 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67031e57-94b5-48c3-96f3-c3ba6ff38668 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:59:00.365133 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67031e57-94b5-48c3-96f3-c3ba6ff38668 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:59:00.365409 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67031e57-94b5-48c3-96f3-c3ba6ff38668 admin admin] PATH_INFO: `/v3/projects/2ef8412eaa9f4cf4a0fd91d0cdfae4b7/users/d869c8094345491a9ac721a73ccac983/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:59:00.380932 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67031e57-94b5-48c3-96f3-c3ba6ff38668 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ef8412eaa9f4cf4a0fd91d0cdfae4b7, user_id=d869c8094345491a9ac721a73ccac983, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:59:00.383607 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67031e57-94b5-48c3-96f3-c3ba6ff38668 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:59:00.397801 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1015/2041] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:59:00 2022] PUT /identity/v3/projects/2ef8412eaa9f4cf4a0fd91d0cdfae4b7/users/d869c8094345491a9ac721a73ccac983/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 03:59:00.401757 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7c52de-1e05-463b-8871-d173716ac76e admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:59:00.411711 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7c52de-1e05-463b-8871-d173716ac76e admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:59:00.468463 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5000e456-32e7-45bf-95f4-400ab8349344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:59:00.469110 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5000e456-32e7-45bf-95f4-400ab8349344 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:59:00.469394 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5000e456-32e7-45bf-95f4-400ab8349344 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:59:00.469673 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5000e456-32e7-45bf-95f4-400ab8349344 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:59:00.470350 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5000e456-32e7-45bf-95f4-400ab8349344 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:59:00.472017 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5000e456-32e7-45bf-95f4-400ab8349344 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:59:00.482795 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1027/2042] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 03:59:00 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 03:59:00.487002 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67031e57-94b5-48c3-96f3-c3ba6ff38668 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:59:00.492874 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:59:00.501755 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67031e57-94b5-48c3-96f3-c3ba6ff38668 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:59:00.506674 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b180a9-d585-427c-9c4e-fc9e575191cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:59:00.507335 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b180a9-d585-427c-9c4e-fc9e575191cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:59:00.507608 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b180a9-d585-427c-9c4e-fc9e575191cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:59:00.507875 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b180a9-d585-427c-9c4e-fc9e575191cb admin admin] PATH_INFO: `/v3/projects/2ef8412eaa9f4cf4a0fd91d0cdfae4b7/users/d869c8094345491a9ac721a73ccac983/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:59:00.509786 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b180a9-d585-427c-9c4e-fc9e575191cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=2ef8412eaa9f4cf4a0fd91d0cdfae4b7, user_id=d869c8094345491a9ac721a73ccac983, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:59:00.512471 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b180a9-d585-427c-9c4e-fc9e575191cb admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:59:00.524020 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1016/2043] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 03:59:00 2022] PUT /identity/v3/projects/2ef8412eaa9f4cf4a0fd91d0cdfae4b7/users/d869c8094345491a9ac721a73ccac983/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 03:59:00.534385 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97565520-fdb1-4ffc-a734-455a85e7d57a None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:59:00.534685 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97565520-fdb1-4ffc-a734-455a85e7d57a None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:59:00.534956 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97565520-fdb1-4ffc-a734-455a85e7d57a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:59:00.567689 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-97565520-fdb1-4ffc-a734-455a85e7d57a None None] MFA Rules not processed for user `d869c8094345491a9ac721a73ccac983`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 03:59:00.568961 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-97565520-fdb1-4ffc-a734-455a85e7d57a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:59:00.640218 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1028/2044] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 03:59:00 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 03:59:10.519188 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=4,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:59:10.637466 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=83,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:59:24.439730 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b180a9-d585-427c-9c4e-fc9e575191cb admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:59:24.441455 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-06b180a9-d585-427c-9c4e-fc9e575191cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:59:24.498191 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b180a9-d585-427c-9c4e-fc9e575191cb admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:59:24.502016 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10800e22-70a9-4f50-9227-57c2c0ee5081 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:59:24.502621 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10800e22-70a9-4f50-9227-57c2c0ee5081 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:59:24.502901 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10800e22-70a9-4f50-9227-57c2c0ee5081 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:59:24.503175 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10800e22-70a9-4f50-9227-57c2c0ee5081 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:59:24.504908 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-10800e22-70a9-4f50-9227-57c2c0ee5081 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:59:24.575431 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10800e22-70a9-4f50-9227-57c2c0ee5081 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:59:24.576884 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10800e22-70a9-4f50-9227-57c2c0ee5081 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:59:24.587845 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1017/2045] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:59:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:59:34.582706 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=54 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 03:59:55.596245 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97565520-fdb1-4ffc-a734-455a85e7d57a None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:59:55.597665 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-97565520-fdb1-4ffc-a734-455a85e7d57a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:59:55.609688 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97565520-fdb1-4ffc-a734-455a85e7d57a None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:59:55.613193 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd46e06-7679-4247-902f-a29859490438 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:59:55.613787 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd46e06-7679-4247-902f-a29859490438 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:59:55.614062 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd46e06-7679-4247-902f-a29859490438 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:59:55.614328 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd46e06-7679-4247-902f-a29859490438 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:59:55.615943 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-0bd46e06-7679-4247-902f-a29859490438 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:59:55.625732 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd46e06-7679-4247-902f-a29859490438 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:59:55.627167 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd46e06-7679-4247-902f-a29859490438 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:59:55.634905 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1029/2046] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:59:55 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 03:59:56.524895 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10800e22-70a9-4f50-9227-57c2c0ee5081 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 03:59:56.526513 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-10800e22-70a9-4f50-9227-57c2c0ee5081 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:59:56.541068 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10800e22-70a9-4f50-9227-57c2c0ee5081 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 03:59:56.546438 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e979c3c3-6eb7-4b97-a402-d866563c53d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 03:59:56.547036 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e979c3c3-6eb7-4b97-a402-d866563c53d6 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 03:59:56.547311 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e979c3c3-6eb7-4b97-a402-d866563c53d6 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 03:59:56.547579 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e979c3c3-6eb7-4b97-a402-d866563c53d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 03:59:56.549282 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e979c3c3-6eb7-4b97-a402-d866563c53d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 03:59:56.617927 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e979c3c3-6eb7-4b97-a402-d866563c53d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 03:59:56.619492 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e979c3c3-6eb7-4b97-a402-d866563c53d6 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 03:59:56.628691 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1018/2047] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 03:59:56 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:00:00.703815 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd46e06-7679-4247-902f-a29859490438 service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:00:00.766064 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd46e06-7679-4247-902f-a29859490438 service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:00:00.771068 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6f6f7e-38a8-4b59-b692-c458ddc21938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:00:00.771679 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6f6f7e-38a8-4b59-b692-c458ddc21938 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:00:00.771978 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6f6f7e-38a8-4b59-b692-c458ddc21938 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:00:00.772248 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6f6f7e-38a8-4b59-b692-c458ddc21938 admin admin] PATH_INFO: `/v3/users/0325cee0036f490c9f732d73a58d128a` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:00:00.773345 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6f6f7e-38a8-4b59-b692-c458ddc21938 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0325cee0036f490c9f732d73a58d128a)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:00:00.775237 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6f6f7e-38a8-4b59-b692-c458ddc21938 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:00:00.821492 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ac6f6f7e-38a8-4b59-b692-c458ddc21938 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0325cee0036f490c9f732d73a58d128a'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:00:00.822062 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ac6f6f7e-38a8-4b59-b692-c458ddc21938 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0325cee0036f490c9f732d73a58d128a'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:00:00.822701 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ac6f6f7e-38a8-4b59-b692-c458ddc21938 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0325cee0036f490c9f732d73a58d128a'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:00:00.835120 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ac6f6f7e-38a8-4b59-b692-c458ddc21938 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0325cee0036f490c9f732d73a58d128a'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:00:00.842369 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ac6f6f7e-38a8-4b59-b692-c458ddc21938 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0325cee0036f490c9f732d73a58d128a'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:00:00.854104 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1030/2048] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 04:00:00 2022] DELETE /identity/v3/users/0325cee0036f490c9f732d73a58d128a => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:00:00.859065 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6010981-550f-4922-a13e-d9347cd9d1d0 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:00:00.859551 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6010981-550f-4922-a13e-d9347cd9d1d0 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:00:00.859656 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6010981-550f-4922-a13e-d9347cd9d1d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:00:00.878805 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-e6010981-550f-4922-a13e-d9347cd9d1d0 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:00:00.880024 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e6010981-550f-4922-a13e-d9347cd9d1d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:00:00.936445 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1019/2049] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:00:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:00:00.945839 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6f6f7e-38a8-4b59-b692-c458ddc21938 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:00:00.947237 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ac6f6f7e-38a8-4b59-b692-c458ddc21938 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:00:01.003076 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6f6f7e-38a8-4b59-b692-c458ddc21938 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:00:01.006883 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aec3961-fbd9-4ade-9849-55a23be65b06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:00:01.007477 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aec3961-fbd9-4ade-9849-55a23be65b06 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:00:01.007751 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aec3961-fbd9-4ade-9849-55a23be65b06 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:00:01.008059 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aec3961-fbd9-4ade-9849-55a23be65b06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:00:01.017036 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aec3961-fbd9-4ade-9849-55a23be65b06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:00:01.018448 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aec3961-fbd9-4ade-9849-55a23be65b06 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:00:01.027731 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1031/2050] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:00:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:00:01.320513 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6010981-550f-4922-a13e-d9347cd9d1d0 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:00:01.321990 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e6010981-550f-4922-a13e-d9347cd9d1d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:00:01.334043 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6010981-550f-4922-a13e-d9347cd9d1d0 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:00:01.338598 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7421212d-ede5-4070-97dd-5d0990f77b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:00:01.339231 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7421212d-ede5-4070-97dd-5d0990f77b24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:00:01.339512 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7421212d-ede5-4070-97dd-5d0990f77b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:00:01.339785 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7421212d-ede5-4070-97dd-5d0990f77b24 admin admin] PATH_INFO: `/v3/projects/264dfc1ce7e242929e8bcd650c248cd3` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:00:01.341020 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7421212d-ede5-4070-97dd-5d0990f77b24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=264dfc1ce7e242929e8bcd650c248cd3)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:00:01.343456 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7421212d-ede5-4070-97dd-5d0990f77b24 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:00:01.393586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7421212d-ede5-4070-97dd-5d0990f77b24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '264dfc1ce7e242929e8bcd650c248cd3', 'request_id': 'req-7421212d-ede5-4070-97dd-5d0990f77b24'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:00:01.399550 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7421212d-ede5-4070-97dd-5d0990f77b24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '264dfc1ce7e242929e8bcd650c248cd3', 'request_id': 'req-7421212d-ede5-4070-97dd-5d0990f77b24'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:00:01.409156 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7421212d-ede5-4070-97dd-5d0990f77b24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '264dfc1ce7e242929e8bcd650c248cd3', 'request_id': 'req-7421212d-ede5-4070-97dd-5d0990f77b24'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:00:01.420778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7421212d-ede5-4070-97dd-5d0990f77b24 admin admin] The token cache is being invalidate because project 264dfc1ce7e242929e8bcd650c248cd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 04:00:01.421062 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7421212d-ede5-4070-97dd-5d0990f77b24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:00:01.421683 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7421212d-ede5-4070-97dd-5d0990f77b24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:00:01.423166 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1020/2051] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 04:00:01 2022] DELETE /identity/v3/projects/264dfc1ce7e242929e8bcd650c248cd3 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:00:01.427195 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aec3961-fbd9-4ade-9849-55a23be65b06 service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:00:01.437098 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aec3961-fbd9-4ade-9849-55a23be65b06 service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:00:01.441865 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:00:01.493948 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f548f86-d238-4781-b074-8ec078c8fe05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:00:01.494567 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f548f86-d238-4781-b074-8ec078c8fe05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:00:01.494844 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f548f86-d238-4781-b074-8ec078c8fe05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:00:01.495112 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f548f86-d238-4781-b074-8ec078c8fe05 admin admin] PATH_INFO: `/v3/users/d869c8094345491a9ac721a73ccac983` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:00:01.496237 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f548f86-d238-4781-b074-8ec078c8fe05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d869c8094345491a9ac721a73ccac983)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:00:01.498144 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f548f86-d238-4781-b074-8ec078c8fe05 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:00:01.548244 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-4f548f86-d238-4781-b074-8ec078c8fe05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd869c8094345491a9ac721a73ccac983'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:00:01.548991 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-4f548f86-d238-4781-b074-8ec078c8fe05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd869c8094345491a9ac721a73ccac983'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:00:01.549582 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-4f548f86-d238-4781-b074-8ec078c8fe05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd869c8094345491a9ac721a73ccac983'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:00:01.561164 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-4f548f86-d238-4781-b074-8ec078c8fe05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd869c8094345491a9ac721a73ccac983'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:00:01.568706 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-4f548f86-d238-4781-b074-8ec078c8fe05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd869c8094345491a9ac721a73ccac983'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:00:01.581007 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1032/2052] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 04:00:01 2022] DELETE /identity/v3/users/d869c8094345491a9ac721a73ccac983 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:00:01.850340 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7421212d-ede5-4070-97dd-5d0990f77b24 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:00:01.851760 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7421212d-ede5-4070-97dd-5d0990f77b24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:00:01.865974 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7421212d-ede5-4070-97dd-5d0990f77b24 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:00:01.873984 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:00:01.925039 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b674e519-c8bb-4d3d-918d-a42a76cf5f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:00:01.925595 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674e519-c8bb-4d3d-918d-a42a76cf5f28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:00:01.925875 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674e519-c8bb-4d3d-918d-a42a76cf5f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:00:01.926145 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674e519-c8bb-4d3d-918d-a42a76cf5f28 admin admin] PATH_INFO: `/v3/projects/2ef8412eaa9f4cf4a0fd91d0cdfae4b7` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:00:01.927243 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b674e519-c8bb-4d3d-918d-a42a76cf5f28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2ef8412eaa9f4cf4a0fd91d0cdfae4b7)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:00:01.929174 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b674e519-c8bb-4d3d-918d-a42a76cf5f28 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:00:01.974396 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b674e519-c8bb-4d3d-918d-a42a76cf5f28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2ef8412eaa9f4cf4a0fd91d0cdfae4b7', 'request_id': 'req-b674e519-c8bb-4d3d-918d-a42a76cf5f28'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:00:01.980326 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b674e519-c8bb-4d3d-918d-a42a76cf5f28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2ef8412eaa9f4cf4a0fd91d0cdfae4b7', 'request_id': 'req-b674e519-c8bb-4d3d-918d-a42a76cf5f28'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:00:01.987518 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b674e519-c8bb-4d3d-918d-a42a76cf5f28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2ef8412eaa9f4cf4a0fd91d0cdfae4b7', 'request_id': 'req-b674e519-c8bb-4d3d-918d-a42a76cf5f28'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:00:01.997601 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b674e519-c8bb-4d3d-918d-a42a76cf5f28 admin admin] The token cache is being invalidate because project 2ef8412eaa9f4cf4a0fd91d0cdfae4b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 04:00:01.998380 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b674e519-c8bb-4d3d-918d-a42a76cf5f28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:00:01.999014 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b674e519-c8bb-4d3d-918d-a42a76cf5f28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:00:02.000745 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1021/2053] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 04:00:01 2022] DELETE /identity/v3/projects/2ef8412eaa9f4cf4a0fd91d0cdfae4b7 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:00:02.008928 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3f5ecc-cbd3-43d4-a0cf-4b8c392cf52b None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:00:02.009459 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3f5ecc-cbd3-43d4-a0cf-4b8c392cf52b None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:00:02.009459 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3f5ecc-cbd3-43d4-a0cf-4b8c392cf52b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:00:02.029544 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-9c3f5ecc-cbd3-43d4-a0cf-4b8c392cf52b None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:00:02.030725 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-9c3f5ecc-cbd3-43d4-a0cf-4b8c392cf52b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:00:02.089620 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1033/2054] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:00:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:00:02.097284 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d643f0-d391-4a29-9576-f1a00b212251 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:00:02.097572 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d643f0-d391-4a29-9576-f1a00b212251 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:00:02.097848 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d643f0-d391-4a29-9576-f1a00b212251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:00:02.117424 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-f1d643f0-d391-4a29-9576-f1a00b212251 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:00:02.118670 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f1d643f0-d391-4a29-9576-f1a00b212251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:00:02.128116 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1022/2055] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:00:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:00:02.132306 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3f5ecc-cbd3-43d4-a0cf-4b8c392cf52b None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:00:02.142406 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3f5ecc-cbd3-43d4-a0cf-4b8c392cf52b None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:00:02.147100 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7d4487-9d21-4023-8fcd-7fb8dc896e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:00:02.147694 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d4487-9d21-4023-8fcd-7fb8dc896e69 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:00:02.147991 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d4487-9d21-4023-8fcd-7fb8dc896e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:00:02.148266 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7d4487-9d21-4023-8fcd-7fb8dc896e69 admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:00:02.148911 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7d4487-9d21-4023-8fcd-7fb8dc896e69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 04:00:02.149208 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7d4487-9d21-4023-8fcd-7fb8dc896e69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:00:02.150783 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7d4487-9d21-4023-8fcd-7fb8dc896e69 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:00:02.165352 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1034/2056] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 04:00:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 04:00:02.167259 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d643f0-d391-4a29-9576-f1a00b212251 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:00:02.176833 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d643f0-d391-4a29-9576-f1a00b212251 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:00:02.182967 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc7e382-1d10-4da4-9987-27d40688bb24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:00:02.183570 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc7e382-1d10-4da4-9987-27d40688bb24 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:00:02.183844 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc7e382-1d10-4da4-9987-27d40688bb24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:00:02.184144 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc7e382-1d10-4da4-9987-27d40688bb24 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:00:02.184780 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc7e382-1d10-4da4-9987-27d40688bb24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:00:02.187240 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc7e382-1d10-4da4-9987-27d40688bb24 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:00:02.219686 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1023/2057] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 04:00:02 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 04:00:02.220915 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7d4487-9d21-4023-8fcd-7fb8dc896e69 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:00:02.230951 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7d4487-9d21-4023-8fcd-7fb8dc896e69 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:00:02.242285 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:00:02.290037 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889fa400-76fb-4c2f-8fc7-604f49c3d291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:00:02.290639 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889fa400-76fb-4c2f-8fc7-604f49c3d291 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:00:02.290934 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889fa400-76fb-4c2f-8fc7-604f49c3d291 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:00:02.291197 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889fa400-76fb-4c2f-8fc7-604f49c3d291 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:00:02.291823 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889fa400-76fb-4c2f-8fc7-604f49c3d291 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:00:02.293762 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889fa400-76fb-4c2f-8fc7-604f49c3d291 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:00:02.314590 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1035/2058] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 04:00:02 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 04:00:02.318549 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc7e382-1d10-4da4-9987-27d40688bb24 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:00:02.328709 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc7e382-1d10-4da4-9987-27d40688bb24 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:00:02.333306 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0d2d4e-cc74-4c8d-8b09-af646682e549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:00:02.333932 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d2d4e-cc74-4c8d-8b09-af646682e549 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:00:02.334211 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d2d4e-cc74-4c8d-8b09-af646682e549 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:00:02.334482 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0d2d4e-cc74-4c8d-8b09-af646682e549 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:00:02.335122 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0d2d4e-cc74-4c8d-8b09-af646682e549 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:00:02.336765 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0d2d4e-cc74-4c8d-8b09-af646682e549 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:00:02.346967 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1024/2059] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:00:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:00:02.351079 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889fa400-76fb-4c2f-8fc7-604f49c3d291 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:00:02.361513 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889fa400-76fb-4c2f-8fc7-604f49c3d291 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:00:02.366194 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d58fd97-ca04-46c3-b9b7-35600d873450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:00:02.366791 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d58fd97-ca04-46c3-b9b7-35600d873450 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:00:02.367073 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d58fd97-ca04-46c3-b9b7-35600d873450 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:00:02.367348 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d58fd97-ca04-46c3-b9b7-35600d873450 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:00:02.368043 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d58fd97-ca04-46c3-b9b7-35600d873450 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:00:02.369659 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d58fd97-ca04-46c3-b9b7-35600d873450 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:00:02.381014 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1036/2060] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:00:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:00:02.384883 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0d2d4e-cc74-4c8d-8b09-af646682e549 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:00:02.394803 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0d2d4e-cc74-4c8d-8b09-af646682e549 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:00:02.399582 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df06856f-92a9-492f-b489-5f252d3590d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:00:02.400264 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df06856f-92a9-492f-b489-5f252d3590d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:00:02.400569 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df06856f-92a9-492f-b489-5f252d3590d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:00:02.400839 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df06856f-92a9-492f-b489-5f252d3590d3 admin admin] PATH_INFO: `/v3/projects/38054306f6bf4fcbb5215710799a5fc3/users/6b04263cfd364bb19da81fa9b1a10c11/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:00:02.415375 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df06856f-92a9-492f-b489-5f252d3590d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38054306f6bf4fcbb5215710799a5fc3, user_id=6b04263cfd364bb19da81fa9b1a10c11, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:00:02.418122 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df06856f-92a9-492f-b489-5f252d3590d3 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:00:02.431570 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1025/2061] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 04:00:02 2022] PUT /identity/v3/projects/38054306f6bf4fcbb5215710799a5fc3/users/6b04263cfd364bb19da81fa9b1a10c11/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:00:02.443141 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9931425c-47a8-42d5-9bb6-53fe94edf2ff None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:00:02.443515 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9931425c-47a8-42d5-9bb6-53fe94edf2ff None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:00:02.443795 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9931425c-47a8-42d5-9bb6-53fe94edf2ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:00:02.471400 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-9931425c-47a8-42d5-9bb6-53fe94edf2ff None None] MFA Rules not processed for user `6b04263cfd364bb19da81fa9b1a10c11`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:00:02.472649 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-9931425c-47a8-42d5-9bb6-53fe94edf2ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:00:02.537787 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1037/2062] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:00:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:00:12.427814 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=73,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:00:12.534729 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=59,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:00:58.745604 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df06856f-92a9-492f-b489-5f252d3590d3 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:00:58.747264 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-df06856f-92a9-492f-b489-5f252d3590d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:00:58.803110 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df06856f-92a9-492f-b489-5f252d3590d3 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:00:58.806713 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa4b4ff-9386-4e35-828c-585aa333eec9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:00:58.807307 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa4b4ff-9386-4e35-828c-585aa333eec9 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:00:58.807582 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa4b4ff-9386-4e35-828c-585aa333eec9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:00:58.807853 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa4b4ff-9386-4e35-828c-585aa333eec9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:00:58.809594 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3fa4b4ff-9386-4e35-828c-585aa333eec9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:00:58.822112 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-3fa4b4ff-9386-4e35-828c-585aa333eec9 service cinder] Could not find project: 2ef8412eaa9f4cf4a0fd91d0cdfae4b7.: keystone.exception.ProjectNotFound: Could not find project: 2ef8412eaa9f4cf4a0fd91d0cdfae4b7. Nov 02 04:00:58.822112 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Nov 02 04:00:58.822112 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Nov 02 04:00:58.822112 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Nov 02 04:00:58.822112 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Nov 02 04:00:58.822112 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Nov 02 04:00:58.822112 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Nov 02 04:00:58.822112 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Nov 02 04:00:58.822112 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Nov 02 04:00:58.822112 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Nov 02 04:00:58.822112 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Nov 02 04:00:58.822112 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Nov 02 04:00:58.822112 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get Nov 02 04:00:58.822112 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') Nov 02 04:00:58.822112 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call Nov 02 04:00:58.823459 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() Nov 02 04:00:58.823459 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data Nov 02 04:00:58.823459 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( Nov 02 04:00:58.823459 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 04:00:58.823459 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 04:00:58.823459 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token Nov 02 04:00:58.823459 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application token = self._validate_token(token_id) Nov 02 04:00:58.823459 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 04:00:58.823459 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 04:00:58.823459 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 04:00:58.823459 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 04:00:58.823459 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 04:00:58.823459 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 04:00:58.823459 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 04:00:58.823459 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 04:00:58.823459 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 04:00:58.823459 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 04:00:58.825099 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 04:00:58.825099 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 04:00:58.825099 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 04:00:58.825099 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 04:00:58.825099 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token Nov 02 04:00:58.825099 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) Nov 02 04:00:58.825099 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint Nov 02 04:00:58.825099 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._validate_token_resources() Nov 02 04:00:58.825099 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources Nov 02 04:00:58.825099 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): Nov 02 04:00:58.825099 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project Nov 02 04:00:58.825099 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( Nov 02 04:00:58.825099 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Nov 02 04:00:58.825099 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Nov 02 04:00:58.825099 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Nov 02 04:00:58.825099 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Nov 02 04:00:58.825099 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Nov 02 04:00:58.826415 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Nov 02 04:00:58.826415 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Nov 02 04:00:58.826415 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Nov 02 04:00:58.826415 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Nov 02 04:00:58.826415 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Nov 02 04:00:58.826415 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Nov 02 04:00:58.826415 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Nov 02 04:00:58.826415 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Nov 02 04:00:58.826415 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Nov 02 04:00:58.826415 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Nov 02 04:00:58.826415 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Nov 02 04:00:58.826415 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Nov 02 04:00:58.826415 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Nov 02 04:00:58.826415 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Nov 02 04:00:58.826415 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Nov 02 04:00:58.826415 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Nov 02 04:00:58.826415 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Nov 02 04:00:58.828528 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 2ef8412eaa9f4cf4a0fd91d0cdfae4b7. Nov 02 04:00:58.828528 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Nov 02 04:00:58.828528 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1026/2063] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:00:58 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 78 msecs (HTTP/1.1 404) 5 headers in 184 bytes (1 switches on core 0) Nov 02 04:00:58.893916 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9931425c-47a8-42d5-9bb6-53fe94edf2ff None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:00:58.903210 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9931425c-47a8-42d5-9bb6-53fe94edf2ff None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:00:58.906724 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cb66ea-39a5-4db9-a773-e13ffb87e682 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:00:58.907344 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb66ea-39a5-4db9-a773-e13ffb87e682 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:00:58.907619 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb66ea-39a5-4db9-a773-e13ffb87e682 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:00:58.907886 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb66ea-39a5-4db9-a773-e13ffb87e682 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:00:58.909744 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-52cb66ea-39a5-4db9-a773-e13ffb87e682 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:00:58.978839 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cb66ea-39a5-4db9-a773-e13ffb87e682 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:00:58.980299 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cb66ea-39a5-4db9-a773-e13ffb87e682 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:00:58.989759 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1038/2064] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:00:58 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:00:59.015927 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa4b4ff-9386-4e35-828c-585aa333eec9 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:00:59.024980 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa4b4ff-9386-4e35-828c-585aa333eec9 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:00:59.028508 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a62ef4d-c757-463b-a1a8-8002f03a4b2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:00:59.029085 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62ef4d-c757-463b-a1a8-8002f03a4b2a service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:00:59.029359 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62ef4d-c757-463b-a1a8-8002f03a4b2a service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:00:59.029626 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a62ef4d-c757-463b-a1a8-8002f03a4b2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:00:59.031507 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0a62ef4d-c757-463b-a1a8-8002f03a4b2a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:00:59.100522 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a62ef4d-c757-463b-a1a8-8002f03a4b2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:00:59.101930 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a62ef4d-c757-463b-a1a8-8002f03a4b2a service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:00:59.111311 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1027/2065] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:00:59 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:00:59.187975 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cb66ea-39a5-4db9-a773-e13ffb87e682 service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:00:59.196252 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cb66ea-39a5-4db9-a773-e13ffb87e682 service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:00:59.199804 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37659759-1e02-41de-8528-ee66cd3f1d6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:00:59.200424 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37659759-1e02-41de-8528-ee66cd3f1d6b service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:00:59.200700 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37659759-1e02-41de-8528-ee66cd3f1d6b service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:00:59.200965 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37659759-1e02-41de-8528-ee66cd3f1d6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:00:59.209197 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37659759-1e02-41de-8528-ee66cd3f1d6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:00:59.210621 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37659759-1e02-41de-8528-ee66cd3f1d6b service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:00:59.220121 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1039/2066] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:00:59 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:01:09.106619 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=57 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:01:09.215409 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=39 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:02:40.074282 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a62ef4d-c757-463b-a1a8-8002f03a4b2a service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:02:40.076378 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0a62ef4d-c757-463b-a1a8-8002f03a4b2a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:02:40.132120 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a62ef4d-c757-463b-a1a8-8002f03a4b2a service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:02:40.135653 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a561a3-1641-4619-8c1c-718436b62fc4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:02:40.136273 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a561a3-1641-4619-8c1c-718436b62fc4 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:02:40.136551 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a561a3-1641-4619-8c1c-718436b62fc4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:02:40.136821 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a561a3-1641-4619-8c1c-718436b62fc4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:02:40.138473 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-62a561a3-1641-4619-8c1c-718436b62fc4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:02:40.205964 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a561a3-1641-4619-8c1c-718436b62fc4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:02:40.207368 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a561a3-1641-4619-8c1c-718436b62fc4 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:02:40.218278 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1028/2067] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:02:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:02:49.023118 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37659759-1e02-41de-8528-ee66cd3f1d6b service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:02:49.024537 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-37659759-1e02-41de-8528-ee66cd3f1d6b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:02:49.162927 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37659759-1e02-41de-8528-ee66cd3f1d6b service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:02:49.167926 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c49672f5-4d5f-4521-91c9-6cd518552bb2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4b9bd741574ddabadd4b7651ebc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:02:49.169068 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49672f5-4d5f-4521-91c9-6cd518552bb2 service placement] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:02:49.169344 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49672f5-4d5f-4521-91c9-6cd518552bb2 service placement] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:02:49.169610 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c49672f5-4d5f-4521-91c9-6cd518552bb2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:02:49.171354 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-c49672f5-4d5f-4521-91c9-6cd518552bb2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:02:49.183024 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49672f5-4d5f-4521-91c9-6cd518552bb2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:02:49.185219 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c49672f5-4d5f-4521-91c9-6cd518552bb2 service placement] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:02:49.196590 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1040/2068] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:02:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:02:50.213259 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=54 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:02:59.190919 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=63 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:02:59.974395 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a561a3-1641-4619-8c1c-718436b62fc4 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:02:59.994496 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a561a3-1641-4619-8c1c-718436b62fc4 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:02:59.998211 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5937c6df-0763-40b5-a7af-18c70eb6925e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:02:59.998816 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5937c6df-0763-40b5-a7af-18c70eb6925e service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:02:59.999362 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5937c6df-0763-40b5-a7af-18c70eb6925e service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:02:59.999636 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5937c6df-0763-40b5-a7af-18c70eb6925e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:03:00.001405 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5937c6df-0763-40b5-a7af-18c70eb6925e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:03:00.050256 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b2f63-ef37-40a1-85e9-3434ec8ae9e2 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:03:00.050574 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b2f63-ef37-40a1-85e9-3434ec8ae9e2 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:03:00.050852 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b2f63-ef37-40a1-85e9-3434ec8ae9e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:03:00.073181 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5937c6df-0763-40b5-a7af-18c70eb6925e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:03:00.074754 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5937c6df-0763-40b5-a7af-18c70eb6925e service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:03:00.078798 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-da0b2f63-ef37-40a1-85e9-3434ec8ae9e2 None None] MFA Rules not processed for user `18355f4225c9427dbb38fde84115d7b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:03:00.080114 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-da0b2f63-ef37-40a1-85e9-3434ec8ae9e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:03:00.083883 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1029/2069] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:02:59 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 04:03:00.138678 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1041/2070] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:03:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:03:00.469202 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5937c6df-0763-40b5-a7af-18c70eb6925e service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:03:00.479259 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5937c6df-0763-40b5-a7af-18c70eb6925e service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:03:00.483251 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ec2fbc-231d-4da1-b7ef-55ada7ea628e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:03:00.483993 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec2fbc-231d-4da1-b7ef-55ada7ea628e service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:03:00.484436 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec2fbc-231d-4da1-b7ef-55ada7ea628e service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:03:00.484728 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ec2fbc-231d-4da1-b7ef-55ada7ea628e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:03:00.486620 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-48ec2fbc-231d-4da1-b7ef-55ada7ea628e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:03:00.496513 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ec2fbc-231d-4da1-b7ef-55ada7ea628e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:03:00.498109 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ec2fbc-231d-4da1-b7ef-55ada7ea628e service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:03:00.506886 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1030/2071] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:03:00 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 04:03:00.835441 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0b2f63-ef37-40a1-85e9-3434ec8ae9e2 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:03:00.836853 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-da0b2f63-ef37-40a1-85e9-3434ec8ae9e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:03:00.851192 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0b2f63-ef37-40a1-85e9-3434ec8ae9e2 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:03:00.856194 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92367720-2384-4afe-a660-f838680640bd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4b9bd741574ddabadd4b7651ebc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:03:00.856797 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92367720-2384-4afe-a660-f838680640bd service placement] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:03:00.857075 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92367720-2384-4afe-a660-f838680640bd service placement] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:03:00.857347 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92367720-2384-4afe-a660-f838680640bd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:03:00.859050 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-92367720-2384-4afe-a660-f838680640bd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:03:00.871977 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92367720-2384-4afe-a660-f838680640bd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:03:00.873656 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92367720-2384-4afe-a660-f838680640bd service placement] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:03:00.884102 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1042/2072] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:03:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:03:03.627838 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ec2fbc-231d-4da1-b7ef-55ada7ea628e service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:03:03.629481 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-48ec2fbc-231d-4da1-b7ef-55ada7ea628e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:03:03.648170 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ec2fbc-231d-4da1-b7ef-55ada7ea628e service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:03:03.654234 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0259f019-5b73-496d-821f-f69c9a895c9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:03:03.654840 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0259f019-5b73-496d-821f-f69c9a895c9f service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:03:03.655121 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0259f019-5b73-496d-821f-f69c9a895c9f service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:03:03.655394 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0259f019-5b73-496d-821f-f69c9a895c9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:03:03.657148 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0259f019-5b73-496d-821f-f69c9a895c9f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:03:03.675492 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0259f019-5b73-496d-821f-f69c9a895c9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:03:03.677510 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0259f019-5b73-496d-821f-f69c9a895c9f service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:03:03.690101 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1031/2073] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:03:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:03:09.032122 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92367720-2384-4afe-a660-f838680640bd service placement] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:03:09.044803 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92367720-2384-4afe-a660-f838680640bd service placement] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:03:09.051063 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d51922-11e1-4006-bf3d-23dd18e9259e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:03:09.051882 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d51922-11e1-4006-bf3d-23dd18e9259e service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:03:09.052296 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d51922-11e1-4006-bf3d-23dd18e9259e service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:03:09.052662 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d51922-11e1-4006-bf3d-23dd18e9259e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:03:09.054494 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-b6d51922-11e1-4006-bf3d-23dd18e9259e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:03:09.067585 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d51922-11e1-4006-bf3d-23dd18e9259e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:03:09.069315 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d51922-11e1-4006-bf3d-23dd18e9259e service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:03:09.079817 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1043/2074] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:03:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:03:12.571726 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0259f019-5b73-496d-821f-f69c9a895c9f service nova] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:03:12.583297 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0259f019-5b73-496d-821f-f69c9a895c9f service nova] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:03:12.588797 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de84fb3d-a918-4746-bf4e-e6e091cdddb1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:03:12.589974 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84fb3d-a918-4746-bf4e-e6e091cdddb1 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:03:12.590254 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84fb3d-a918-4746-bf4e-e6e091cdddb1 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:03:12.590525 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84fb3d-a918-4746-bf4e-e6e091cdddb1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:03:12.592274 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-de84fb3d-a918-4746-bf4e-e6e091cdddb1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:03:12.604339 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de84fb3d-a918-4746-bf4e-e6e091cdddb1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:03:12.605952 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de84fb3d-a918-4746-bf4e-e6e091cdddb1 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:03:12.616701 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1032/2075] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:03:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:03:19.075342 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=40 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:03:22.612097 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=51 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:03:43.283691 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58be0f-16bf-4281-8fd0-a28bae1d3aae None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:03:43.284342 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58be0f-16bf-4281-8fd0-a28bae1d3aae None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:03:43.284342 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d58be0f-16bf-4281-8fd0-a28bae1d3aae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:03:43.311959 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-6d58be0f-16bf-4281-8fd0-a28bae1d3aae None None] MFA Rules not processed for user `7984dc1d452a4c998b44e6d26a299df8`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:03:43.313182 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-6d58be0f-16bf-4281-8fd0-a28bae1d3aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:03:43.378268 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1044/2076] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:03:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:03:53.375203 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=33 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:04:31.540121 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de84fb3d-a918-4746-bf4e-e6e091cdddb1 service nova] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:04:31.567369 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de84fb3d-a918-4746-bf4e-e6e091cdddb1 service nova] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:04:31.573094 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cfe7f1-35f0-4362-8052-ea228423c365 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:04:31.573727 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cfe7f1-35f0-4362-8052-ea228423c365 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:04:31.574010 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cfe7f1-35f0-4362-8052-ea228423c365 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:04:31.574286 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cfe7f1-35f0-4362-8052-ea228423c365 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:04:31.576032 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f5cfe7f1-35f0-4362-8052-ea228423c365 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:04:31.586872 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cfe7f1-35f0-4362-8052-ea228423c365 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:04:31.588471 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cfe7f1-35f0-4362-8052-ea228423c365 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:04:31.597975 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1033/2077] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:04:31 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:04:41.593918 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=13 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:04:48.889862 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d58be0f-16bf-4281-8fd0-a28bae1d3aae None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:04:48.908843 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d58be0f-16bf-4281-8fd0-a28bae1d3aae None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:04:48.912356 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5ae891-63c5-4490-baf3-5838617b338e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:04:48.912944 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ae891-63c5-4490-baf3-5838617b338e service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:04:48.913220 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ae891-63c5-4490-baf3-5838617b338e service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:04:48.913488 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5ae891-63c5-4490-baf3-5838617b338e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:04:48.915279 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-af5ae891-63c5-4490-baf3-5838617b338e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:04:48.942209 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5ae891-63c5-4490-baf3-5838617b338e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:04:48.943623 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5ae891-63c5-4490-baf3-5838617b338e service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:04:48.954335 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1045/2078] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:04:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:04:58.584565 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cfe7f1-35f0-4362-8052-ea228423c365 service nova] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:04:58.585968 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f5cfe7f1-35f0-4362-8052-ea228423c365 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:04:58.597881 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cfe7f1-35f0-4362-8052-ea228423c365 service nova] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:04:58.602040 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae3827c-dc95-45d0-94fc-caab2eacf7b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:04:58.602626 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae3827c-dc95-45d0-94fc-caab2eacf7b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:04:58.602902 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae3827c-dc95-45d0-94fc-caab2eacf7b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:04:58.603173 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae3827c-dc95-45d0-94fc-caab2eacf7b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:04:58.604841 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-eae3827c-dc95-45d0-94fc-caab2eacf7b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:04:58.613947 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae3827c-dc95-45d0-94fc-caab2eacf7b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:04:58.615358 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae3827c-dc95-45d0-94fc-caab2eacf7b1 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:04:58.623128 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1034/2079] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:04:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:04:58.949623 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=15 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:05:00.680292 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5ae891-63c5-4490-baf3-5838617b338e service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:05:00.699361 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5ae891-63c5-4490-baf3-5838617b338e service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:05:00.703127 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd552623-f5e6-428d-b683-d5b7a5af2576 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:05:00.703719 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd552623-f5e6-428d-b683-d5b7a5af2576 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:05:00.704032 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd552623-f5e6-428d-b683-d5b7a5af2576 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:05:00.704294 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd552623-f5e6-428d-b683-d5b7a5af2576 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:05:00.706007 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-bd552623-f5e6-428d-b683-d5b7a5af2576 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:05:00.725808 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd552623-f5e6-428d-b683-d5b7a5af2576 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:05:00.727213 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd552623-f5e6-428d-b683-d5b7a5af2576 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:05:00.746119 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1046/2080] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:05:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:05:01.348271 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae3827c-dc95-45d0-94fc-caab2eacf7b1 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:05:01.356917 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae3827c-dc95-45d0-94fc-caab2eacf7b1 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:05:01.360536 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56540677-04bd-4f4f-a744-161f419c80f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:05:01.361129 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56540677-04bd-4f4f-a744-161f419c80f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:05:01.361406 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56540677-04bd-4f4f-a744-161f419c80f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:05:01.361680 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56540677-04bd-4f4f-a744-161f419c80f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:05:01.363427 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-56540677-04bd-4f4f-a744-161f419c80f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:05:01.428386 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56540677-04bd-4f4f-a744-161f419c80f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:05:01.429845 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56540677-04bd-4f4f-a744-161f419c80f2 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:05:01.440980 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1035/2081] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:05:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:05:04.722284 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd552623-f5e6-428d-b683-d5b7a5af2576 service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:05:04.733730 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd552623-f5e6-428d-b683-d5b7a5af2576 service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:05:04.738661 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd16f35f-1143-4de8-afb4-358a4337e935 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4b9bd741574ddabadd4b7651ebc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:05:04.739262 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd16f35f-1143-4de8-afb4-358a4337e935 service placement] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:05:04.739538 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd16f35f-1143-4de8-afb4-358a4337e935 service placement] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:05:04.739809 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd16f35f-1143-4de8-afb4-358a4337e935 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:05:04.741551 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-dd16f35f-1143-4de8-afb4-358a4337e935 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:05:04.753898 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd16f35f-1143-4de8-afb4-358a4337e935 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:05:04.755479 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd16f35f-1143-4de8-afb4-358a4337e935 service placement] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:05:04.766687 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1047/2082] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:05:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:05:04.777239 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56540677-04bd-4f4f-a744-161f419c80f2 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:05:04.779422 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-56540677-04bd-4f4f-a744-161f419c80f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:05:04.829387 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56540677-04bd-4f4f-a744-161f419c80f2 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:05:04.834066 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87aaa7ea-209d-4dfb-a654-b80397930f5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:05:04.834725 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aaa7ea-209d-4dfb-a654-b80397930f5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:05:04.835003 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aaa7ea-209d-4dfb-a654-b80397930f5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:05:04.835275 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87aaa7ea-209d-4dfb-a654-b80397930f5d admin admin] PATH_INFO: `/v3/users/f16c6a2cd8594037ad47475ed1b3161c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:05:04.836444 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aaa7ea-209d-4dfb-a654-b80397930f5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f16c6a2cd8594037ad47475ed1b3161c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:05:04.838385 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87aaa7ea-209d-4dfb-a654-b80397930f5d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:05:04.882574 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-87aaa7ea-209d-4dfb-a654-b80397930f5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f16c6a2cd8594037ad47475ed1b3161c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:05:04.890121 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-87aaa7ea-209d-4dfb-a654-b80397930f5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f16c6a2cd8594037ad47475ed1b3161c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:05:04.890717 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-87aaa7ea-209d-4dfb-a654-b80397930f5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f16c6a2cd8594037ad47475ed1b3161c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:05:04.902340 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-87aaa7ea-209d-4dfb-a654-b80397930f5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f16c6a2cd8594037ad47475ed1b3161c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:05:04.909359 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-87aaa7ea-209d-4dfb-a654-b80397930f5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f16c6a2cd8594037ad47475ed1b3161c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:05:04.915091 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1036/2083] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 04:05:04 2022] DELETE /identity/v3/users/f16c6a2cd8594037ad47475ed1b3161c => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:05:05.090032 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd16f35f-1143-4de8-afb4-358a4337e935 service placement] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:05:05.109350 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd16f35f-1143-4de8-afb4-358a4337e935 service placement] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:05:05.171643 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9792748-ed9b-4a52-b5c5-5f2696fff088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:05:05.172302 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9792748-ed9b-4a52-b5c5-5f2696fff088 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:05:05.172577 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9792748-ed9b-4a52-b5c5-5f2696fff088 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:05:05.172848 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9792748-ed9b-4a52-b5c5-5f2696fff088 admin admin] PATH_INFO: `/v3/projects/d56f6d222a4d4411b8f4583c61f53129` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:05:05.182927 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9792748-ed9b-4a52-b5c5-5f2696fff088 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d56f6d222a4d4411b8f4583c61f53129)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:05:05.185510 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9792748-ed9b-4a52-b5c5-5f2696fff088 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:05:05.230520 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b9792748-ed9b-4a52-b5c5-5f2696fff088 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd56f6d222a4d4411b8f4583c61f53129', 'request_id': 'req-b9792748-ed9b-4a52-b5c5-5f2696fff088'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:05:05.236504 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b9792748-ed9b-4a52-b5c5-5f2696fff088 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd56f6d222a4d4411b8f4583c61f53129', 'request_id': 'req-b9792748-ed9b-4a52-b5c5-5f2696fff088'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:05:05.243046 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b9792748-ed9b-4a52-b5c5-5f2696fff088 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd56f6d222a4d4411b8f4583c61f53129', 'request_id': 'req-b9792748-ed9b-4a52-b5c5-5f2696fff088'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:05:05.255895 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b9792748-ed9b-4a52-b5c5-5f2696fff088 admin admin] The token cache is being invalidate because project d56f6d222a4d4411b8f4583c61f53129 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 04:05:05.256213 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b9792748-ed9b-4a52-b5c5-5f2696fff088 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:05:05.256827 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b9792748-ed9b-4a52-b5c5-5f2696fff088 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:05:05.258294 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1048/2084] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 04:05:05 2022] DELETE /identity/v3/projects/d56f6d222a4d4411b8f4583c61f53129 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:05:05.264704 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b3ed53-68f0-4c9f-b172-3ea014fc7e34 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:05:05.264992 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b3ed53-68f0-4c9f-b172-3ea014fc7e34 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:05:05.265265 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b3ed53-68f0-4c9f-b172-3ea014fc7e34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:05:05.284214 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-84b3ed53-68f0-4c9f-b172-3ea014fc7e34 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:05:05.285393 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-84b3ed53-68f0-4c9f-b172-3ea014fc7e34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:05:05.304051 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:05:05.346935 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1037/2085] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:05:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:05:05.354749 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a9f187-95a0-481d-aa68-a19c530f93ba None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:05:05.355038 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a9f187-95a0-481d-aa68-a19c530f93ba None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:05:05.355312 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a9f187-95a0-481d-aa68-a19c530f93ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:05:05.375983 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-a1a9f187-95a0-481d-aa68-a19c530f93ba None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:05:05.377228 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a1a9f187-95a0-481d-aa68-a19c530f93ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:05:05.385102 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1049/2086] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:05:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:05:05.389244 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b3ed53-68f0-4c9f-b172-3ea014fc7e34 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:05:05.398823 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b3ed53-68f0-4c9f-b172-3ea014fc7e34 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:05:05.403237 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2556b2a1-1532-4086-a857-853188c56960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:05:05.403824 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2556b2a1-1532-4086-a857-853188c56960 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:05:05.404231 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2556b2a1-1532-4086-a857-853188c56960 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:05:05.404503 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2556b2a1-1532-4086-a857-853188c56960 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:05:05.405138 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2556b2a1-1532-4086-a857-853188c56960 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 04:05:05.405544 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2556b2a1-1532-4086-a857-853188c56960 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:05:05.407129 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2556b2a1-1532-4086-a857-853188c56960 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:05:05.418324 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1038/2087] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 04:05:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 04:05:05.422976 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a9f187-95a0-481d-aa68-a19c530f93ba None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:05:05.432341 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a9f187-95a0-481d-aa68-a19c530f93ba None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:05:05.437646 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7716f2ea-afeb-4d36-965a-ea99187015a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:05:05.438241 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7716f2ea-afeb-4d36-965a-ea99187015a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:05:05.438522 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7716f2ea-afeb-4d36-965a-ea99187015a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:05:05.438798 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7716f2ea-afeb-4d36-965a-ea99187015a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:05:05.439442 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7716f2ea-afeb-4d36-965a-ea99187015a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:05:05.441368 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7716f2ea-afeb-4d36-965a-ea99187015a8 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:05:05.461834 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:05:05.472191 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1050/2088] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 04:05:05 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:05:05.473121 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2556b2a1-1532-4086-a857-853188c56960 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:05:05.483658 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2556b2a1-1532-4086-a857-853188c56960 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:05:05.538985 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf60a905-90a0-4822-8a4c-08990f0b061e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:05:05.539643 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf60a905-90a0-4822-8a4c-08990f0b061e admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:05:05.539918 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf60a905-90a0-4822-8a4c-08990f0b061e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:05:05.540217 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf60a905-90a0-4822-8a4c-08990f0b061e admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:05:05.540880 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf60a905-90a0-4822-8a4c-08990f0b061e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:05:05.542777 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf60a905-90a0-4822-8a4c-08990f0b061e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:05:05.564107 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1039/2089] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 04:05:05 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:05:05.568214 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7716f2ea-afeb-4d36-965a-ea99187015a8 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:05:05.578452 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7716f2ea-afeb-4d36-965a-ea99187015a8 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:05:05.583070 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acadb220-1c8f-4838-a5cb-ed50cdc51a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:05:05.583666 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acadb220-1c8f-4838-a5cb-ed50cdc51a34 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:05:05.583944 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acadb220-1c8f-4838-a5cb-ed50cdc51a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:05:05.584245 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acadb220-1c8f-4838-a5cb-ed50cdc51a34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:05:05.584880 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acadb220-1c8f-4838-a5cb-ed50cdc51a34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:05:05.586463 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acadb220-1c8f-4838-a5cb-ed50cdc51a34 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:05:05.596759 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1051/2090] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:05:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:05:05.600665 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf60a905-90a0-4822-8a4c-08990f0b061e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:05:05.610928 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf60a905-90a0-4822-8a4c-08990f0b061e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:05:05.615437 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6da389-8c85-4348-a14a-6aad4adf263a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:05:05.616050 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6da389-8c85-4348-a14a-6aad4adf263a admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:05:05.616331 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6da389-8c85-4348-a14a-6aad4adf263a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:05:05.616603 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6da389-8c85-4348-a14a-6aad4adf263a admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:05:05.617228 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6da389-8c85-4348-a14a-6aad4adf263a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:05:05.618810 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6da389-8c85-4348-a14a-6aad4adf263a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:05:05.629390 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1040/2091] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:05:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 04:05:05.633422 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acadb220-1c8f-4838-a5cb-ed50cdc51a34 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:05:05.643743 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acadb220-1c8f-4838-a5cb-ed50cdc51a34 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:05:05.648442 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7557e899-2a16-4759-84fe-6ef9d167da61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:05:05.649193 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7557e899-2a16-4759-84fe-6ef9d167da61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:05:05.649417 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7557e899-2a16-4759-84fe-6ef9d167da61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:05:05.649704 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7557e899-2a16-4759-84fe-6ef9d167da61 admin admin] PATH_INFO: `/v3/projects/acc7aac3a9ba4795ab6885d94f7d6cef/users/5fcb338fe09440598db96cef0a532a3f/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:05:05.664874 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7557e899-2a16-4759-84fe-6ef9d167da61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=acc7aac3a9ba4795ab6885d94f7d6cef, user_id=5fcb338fe09440598db96cef0a532a3f, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:05:05.667570 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7557e899-2a16-4759-84fe-6ef9d167da61 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:05:05.681722 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1052/2092] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 04:05:05 2022] PUT /identity/v3/projects/acc7aac3a9ba4795ab6885d94f7d6cef/users/5fcb338fe09440598db96cef0a532a3f/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:05:05.693763 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65bb942-589a-4b8c-b3e5-143717defcbc None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:05:05.694044 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65bb942-589a-4b8c-b3e5-143717defcbc None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:05:05.694317 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65bb942-589a-4b8c-b3e5-143717defcbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:05:05.721710 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-b65bb942-589a-4b8c-b3e5-143717defcbc None None] MFA Rules not processed for user `5fcb338fe09440598db96cef0a532a3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:05:05.722846 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b65bb942-589a-4b8c-b3e5-143717defcbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:05:05.785348 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1041/2093] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:05:05 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:05:05.789840 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7557e899-2a16-4759-84fe-6ef9d167da61 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:05:05.851070 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7557e899-2a16-4759-84fe-6ef9d167da61 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:05:05.855623 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4860842-b94c-41e4-9435-4ddf1922a3ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:05:05.856237 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4860842-b94c-41e4-9435-4ddf1922a3ca admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:05:05.856513 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4860842-b94c-41e4-9435-4ddf1922a3ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:05:05.856827 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4860842-b94c-41e4-9435-4ddf1922a3ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:05:05.857451 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4860842-b94c-41e4-9435-4ddf1922a3ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:05:05.859324 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4860842-b94c-41e4-9435-4ddf1922a3ca admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:05:05.884855 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1053/2094] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 04:05:05 2022] POST /identity/v3/projects => generated 381 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:05:05.889499 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65bb942-589a-4b8c-b3e5-143717defcbc None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:05:05.900402 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65bb942-589a-4b8c-b3e5-143717defcbc None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:05:05.917100 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:05:05.957887 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3624a787-4a6b-4a2b-9c64-3a22f7836568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:05:05.958483 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624a787-4a6b-4a2b-9c64-3a22f7836568 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:05:05.958757 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624a787-4a6b-4a2b-9c64-3a22f7836568 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:05:05.959024 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3624a787-4a6b-4a2b-9c64-3a22f7836568 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:05:05.959656 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3624a787-4a6b-4a2b-9c64-3a22f7836568 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:05:05.961577 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3624a787-4a6b-4a2b-9c64-3a22f7836568 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:05:05.982710 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1042/2095] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 04:05:05 2022] POST /identity/v3/users => generated 297 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:05:05.986634 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4860842-b94c-41e4-9435-4ddf1922a3ca admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:05:05.996773 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4860842-b94c-41e4-9435-4ddf1922a3ca admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:05:06.001684 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffe5dde-999f-4827-925e-b85b057577ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:05:06.002276 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffe5dde-999f-4827-925e-b85b057577ab admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:05:06.002555 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffe5dde-999f-4827-925e-b85b057577ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:05:06.002842 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffe5dde-999f-4827-925e-b85b057577ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:05:06.003463 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffe5dde-999f-4827-925e-b85b057577ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:05:06.005090 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffe5dde-999f-4827-925e-b85b057577ab admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:05:06.016709 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1054/2096] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:05:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:05:06.020932 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3624a787-4a6b-4a2b-9c64-3a22f7836568 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:05:06.031560 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3624a787-4a6b-4a2b-9c64-3a22f7836568 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:05:06.036188 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d550c5f6-02ec-493f-b1d1-097a83f49507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:05:06.036836 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d550c5f6-02ec-493f-b1d1-097a83f49507 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:05:06.037112 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d550c5f6-02ec-493f-b1d1-097a83f49507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:05:06.037402 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d550c5f6-02ec-493f-b1d1-097a83f49507 admin admin] PATH_INFO: `/v3/projects/33149bbc2fba427d92215279e0d5e5e7/users/5bc48f1451c64aff97cba01ca7b00568/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:05:06.051750 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d550c5f6-02ec-493f-b1d1-097a83f49507 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33149bbc2fba427d92215279e0d5e5e7, user_id=5bc48f1451c64aff97cba01ca7b00568, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:05:06.054424 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d550c5f6-02ec-493f-b1d1-097a83f49507 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:05:06.066872 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1043/2097] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 04:05:06 2022] PUT /identity/v3/projects/33149bbc2fba427d92215279e0d5e5e7/users/5bc48f1451c64aff97cba01ca7b00568/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:05:06.070801 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffe5dde-999f-4827-925e-b85b057577ab admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:05:06.081164 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffe5dde-999f-4827-925e-b85b057577ab admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:05:06.136866 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef363416-37e8-4e2a-acc7-e6ca4cc17184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:05:06.137399 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef363416-37e8-4e2a-acc7-e6ca4cc17184 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:05:06.137680 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef363416-37e8-4e2a-acc7-e6ca4cc17184 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:05:06.138484 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef363416-37e8-4e2a-acc7-e6ca4cc17184 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:05:06.139118 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef363416-37e8-4e2a-acc7-e6ca4cc17184 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:05:06.140746 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef363416-37e8-4e2a-acc7-e6ca4cc17184 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:05:06.150331 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1055/2098] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:05:06 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:05:06.155087 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d550c5f6-02ec-493f-b1d1-097a83f49507 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:05:06.165098 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d550c5f6-02ec-493f-b1d1-097a83f49507 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:05:06.169633 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6bfaf4-98f9-48de-9f4e-a3fc9895e9cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:05:06.170269 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6bfaf4-98f9-48de-9f4e-a3fc9895e9cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:05:06.170545 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6bfaf4-98f9-48de-9f4e-a3fc9895e9cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:05:06.170815 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6bfaf4-98f9-48de-9f4e-a3fc9895e9cd admin admin] PATH_INFO: `/v3/projects/33149bbc2fba427d92215279e0d5e5e7/users/5bc48f1451c64aff97cba01ca7b00568/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:05:06.172788 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6bfaf4-98f9-48de-9f4e-a3fc9895e9cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=33149bbc2fba427d92215279e0d5e5e7, user_id=5bc48f1451c64aff97cba01ca7b00568, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:05:06.175392 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6bfaf4-98f9-48de-9f4e-a3fc9895e9cd admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:05:06.185798 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1044/2099] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 04:05:06 2022] PUT /identity/v3/projects/33149bbc2fba427d92215279e0d5e5e7/users/5bc48f1451c64aff97cba01ca7b00568/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:05:06.195712 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1db49-a156-4a19-903a-fcee1c1ba1b1 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:05:06.196028 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1db49-a156-4a19-903a-fcee1c1ba1b1 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:05:06.196304 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1db49-a156-4a19-903a-fcee1c1ba1b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:05:06.223906 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-fdc1db49-a156-4a19-903a-fcee1c1ba1b1 None None] MFA Rules not processed for user `5bc48f1451c64aff97cba01ca7b00568`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:05:06.225136 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-fdc1db49-a156-4a19-903a-fcee1c1ba1b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:05:06.249015 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:05:06.292695 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1056/2100] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:05:06 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:05:16.181625 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=35,keystone:INSERT=5 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:05:16.289789 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=18 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:05:38.106553 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6bfaf4-98f9-48de-9f4e-a3fc9895e9cd admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:05:38.108866 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-cc6bfaf4-98f9-48de-9f4e-a3fc9895e9cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:05:38.197986 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6bfaf4-98f9-48de-9f4e-a3fc9895e9cd admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:05:38.203470 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060ba6c8-8905-47c7-ab30-60220185347d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:05:38.204131 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ba6c8-8905-47c7-ab30-60220185347d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:05:38.204413 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ba6c8-8905-47c7-ab30-60220185347d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:05:38.204687 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060ba6c8-8905-47c7-ab30-60220185347d admin admin] PATH_INFO: `/v3/users/1151a2dd24604c98accfa0821c0d5b04` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:05:38.218658 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060ba6c8-8905-47c7-ab30-60220185347d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1151a2dd24604c98accfa0821c0d5b04)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:05:38.220795 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060ba6c8-8905-47c7-ab30-60220185347d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:05:38.228592 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc1db49-a156-4a19-903a-fcee1c1ba1b1 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:05:38.230551 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-fdc1db49-a156-4a19-903a-fcee1c1ba1b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:05:38.268696 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-060ba6c8-8905-47c7-ab30-60220185347d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1151a2dd24604c98accfa0821c0d5b04'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:05:38.275854 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-060ba6c8-8905-47c7-ab30-60220185347d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1151a2dd24604c98accfa0821c0d5b04'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:05:38.276554 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-060ba6c8-8905-47c7-ab30-60220185347d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1151a2dd24604c98accfa0821c0d5b04'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:05:38.289381 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-060ba6c8-8905-47c7-ab30-60220185347d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1151a2dd24604c98accfa0821c0d5b04'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:05:38.292993 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc1db49-a156-4a19-903a-fcee1c1ba1b1 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:05:38.297182 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e89e9df-b40b-4c01-8350-17b3e9cd0a93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:05:38.297410 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-060ba6c8-8905-47c7-ab30-60220185347d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1151a2dd24604c98accfa0821c0d5b04'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:05:38.297872 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e89e9df-b40b-4c01-8350-17b3e9cd0a93 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:05:38.298178 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e89e9df-b40b-4c01-8350-17b3e9cd0a93 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:05:38.298459 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e89e9df-b40b-4c01-8350-17b3e9cd0a93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:05:38.300308 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-9e89e9df-b40b-4c01-8350-17b3e9cd0a93 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:05:38.303534 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1045/2101] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 04:05:38 2022] DELETE /identity/v3/users/1151a2dd24604c98accfa0821c0d5b04 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:05:38.307839 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e20e6eb-134b-4017-9fac-e7899502958e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:05:38.308222 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e20e6eb-134b-4017-9fac-e7899502958e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:05:38.308499 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e20e6eb-134b-4017-9fac-e7899502958e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:05:38.315684 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e89e9df-b40b-4c01-8350-17b3e9cd0a93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:05:38.317179 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e89e9df-b40b-4c01-8350-17b3e9cd0a93 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:05:38.328318 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1057/2102] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:05:38 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 04:05:38.336943 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-5e20e6eb-134b-4017-9fac-e7899502958e None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:05:38.338143 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5e20e6eb-134b-4017-9fac-e7899502958e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:05:38.402181 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1046/2103] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:05:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 04:05:38.413307 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e89e9df-b40b-4c01-8350-17b3e9cd0a93 service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:05:38.414759 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-9e89e9df-b40b-4c01-8350-17b3e9cd0a93 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:05:38.484497 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e89e9df-b40b-4c01-8350-17b3e9cd0a93 service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:05:38.532851 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a15c293-eb97-4b79-a123-a4c9f519331b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:05:38.533454 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a15c293-eb97-4b79-a123-a4c9f519331b service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:05:38.533729 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a15c293-eb97-4b79-a123-a4c9f519331b service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:05:38.533999 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a15c293-eb97-4b79-a123-a4c9f519331b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:05:38.543153 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a15c293-eb97-4b79-a123-a4c9f519331b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:05:38.544525 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a15c293-eb97-4b79-a123-a4c9f519331b service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:05:38.553823 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1058/2104] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:05:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 04:05:38.817622 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e20e6eb-134b-4017-9fac-e7899502958e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:05:38.819181 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5e20e6eb-134b-4017-9fac-e7899502958e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:05:38.826479 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:05:38.833862 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e20e6eb-134b-4017-9fac-e7899502958e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:05:38.840434 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a09895-bf91-4bb3-afab-4dc47edd7562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:05:38.841076 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a09895-bf91-4bb3-afab-4dc47edd7562 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:05:38.841363 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a09895-bf91-4bb3-afab-4dc47edd7562 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:05:38.841620 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a09895-bf91-4bb3-afab-4dc47edd7562 admin admin] PATH_INFO: `/v3/projects/a04e470e9b6343febac95c1b1d9fb47c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:05:38.852655 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a09895-bf91-4bb3-afab-4dc47edd7562 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a04e470e9b6343febac95c1b1d9fb47c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:05:38.854737 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a09895-bf91-4bb3-afab-4dc47edd7562 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:05:38.900084 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-61a09895-bf91-4bb3-afab-4dc47edd7562 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a04e470e9b6343febac95c1b1d9fb47c', 'request_id': 'req-61a09895-bf91-4bb3-afab-4dc47edd7562'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:05:38.906504 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-61a09895-bf91-4bb3-afab-4dc47edd7562 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a04e470e9b6343febac95c1b1d9fb47c', 'request_id': 'req-61a09895-bf91-4bb3-afab-4dc47edd7562'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:05:38.913951 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-61a09895-bf91-4bb3-afab-4dc47edd7562 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a04e470e9b6343febac95c1b1d9fb47c', 'request_id': 'req-61a09895-bf91-4bb3-afab-4dc47edd7562'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:05:38.924853 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-61a09895-bf91-4bb3-afab-4dc47edd7562 admin admin] The token cache is being invalidate because project a04e470e9b6343febac95c1b1d9fb47c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 04:05:38.925249 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-61a09895-bf91-4bb3-afab-4dc47edd7562 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:05:38.926010 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-61a09895-bf91-4bb3-afab-4dc47edd7562 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:05:38.927718 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1047/2105] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 04:05:38 2022] DELETE /identity/v3/projects/a04e470e9b6343febac95c1b1d9fb47c => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:05:48.549680 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=77 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:05:48.921734 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=30,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:06:47.164687 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a15c293-eb97-4b79-a123-a4c9f519331b service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:06:47.166139 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-1a15c293-eb97-4b79-a123-a4c9f519331b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:06:47.194074 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a09895-bf91-4bb3-afab-4dc47edd7562 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:06:47.195544 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-61a09895-bf91-4bb3-afab-4dc47edd7562 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:06:47.227177 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a15c293-eb97-4b79-a123-a4c9f519331b service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:06:47.256759 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a09895-bf91-4bb3-afab-4dc47edd7562 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:06:47.261639 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c12c7f1-9a52-4aed-a042-8aa889db58b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:06:47.262353 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c12c7f1-9a52-4aed-a042-8aa889db58b1 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:06:47.262626 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c12c7f1-9a52-4aed-a042-8aa889db58b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:06:47.262896 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c12c7f1-9a52-4aed-a042-8aa889db58b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:06:47.265906 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5c12c7f1-9a52-4aed-a042-8aa889db58b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:06:47.283040 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e296ce3-8f9f-49ca-8572-86125c678895 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:06:47.283874 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e296ce3-8f9f-49ca-8572-86125c678895 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:06:47.284253 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e296ce3-8f9f-49ca-8572-86125c678895 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:06:47.284625 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e296ce3-8f9f-49ca-8572-86125c678895 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:06:47.286468 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-7e296ce3-8f9f-49ca-8572-86125c678895 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:06:47.333180 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c12c7f1-9a52-4aed-a042-8aa889db58b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:06:47.334618 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c12c7f1-9a52-4aed-a042-8aa889db58b1 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:06:47.343623 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1048/2106] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:06:47 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 04:06:47.360448 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e296ce3-8f9f-49ca-8572-86125c678895 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:06:47.361953 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e296ce3-8f9f-49ca-8572-86125c678895 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:06:47.382115 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1059/2107] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:06:47 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:06:47.541149 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c12c7f1-9a52-4aed-a042-8aa889db58b1 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:06:47.542821 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5c12c7f1-9a52-4aed-a042-8aa889db58b1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:06:47.634111 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c12c7f1-9a52-4aed-a042-8aa889db58b1 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:06:47.638965 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52365c2-89e5-4b47-b5e6-53a72cb0f1ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:06:47.639598 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52365c2-89e5-4b47-b5e6-53a72cb0f1ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:06:47.639893 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52365c2-89e5-4b47-b5e6-53a72cb0f1ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:06:47.640736 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52365c2-89e5-4b47-b5e6-53a72cb0f1ea admin admin] PATH_INFO: `/v3/users/5fcb338fe09440598db96cef0a532a3f` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:06:47.641877 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52365c2-89e5-4b47-b5e6-53a72cb0f1ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fcb338fe09440598db96cef0a532a3f)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:06:47.643848 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52365c2-89e5-4b47-b5e6-53a72cb0f1ea admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:06:47.715149 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f52365c2-89e5-4b47-b5e6-53a72cb0f1ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fcb338fe09440598db96cef0a532a3f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:06:47.726524 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f52365c2-89e5-4b47-b5e6-53a72cb0f1ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fcb338fe09440598db96cef0a532a3f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:06:47.728414 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f52365c2-89e5-4b47-b5e6-53a72cb0f1ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fcb338fe09440598db96cef0a532a3f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:06:47.745888 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:06:47.777802 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f52365c2-89e5-4b47-b5e6-53a72cb0f1ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fcb338fe09440598db96cef0a532a3f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:06:47.789875 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f52365c2-89e5-4b47-b5e6-53a72cb0f1ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fcb338fe09440598db96cef0a532a3f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:06:47.804821 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1049/2108] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 04:06:47 2022] DELETE /identity/v3/users/5fcb338fe09440598db96cef0a532a3f => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:06:47.810278 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb8cb2-bb5e-4500-a710-f24a59ac361d None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:06:47.810694 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb8cb2-bb5e-4500-a710-f24a59ac361d None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:06:47.810851 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb8cb2-bb5e-4500-a710-f24a59ac361d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:06:47.836903 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-e9fb8cb2-bb5e-4500-a710-f24a59ac361d None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:06:47.838346 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e9fb8cb2-bb5e-4500-a710-f24a59ac361d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:06:47.888121 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:06:47.900605 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1060/2109] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:06:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:06:47.910451 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52365c2-89e5-4b47-b5e6-53a72cb0f1ea admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:06:47.911886 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f52365c2-89e5-4b47-b5e6-53a72cb0f1ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:06:47.969290 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52365c2-89e5-4b47-b5e6-53a72cb0f1ea admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:06:47.972925 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac02667-5053-4698-b32d-ad92350f4f7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:06:47.973534 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac02667-5053-4698-b32d-ad92350f4f7f service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:06:47.973812 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac02667-5053-4698-b32d-ad92350f4f7f service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:06:47.974085 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac02667-5053-4698-b32d-ad92350f4f7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:06:47.982878 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac02667-5053-4698-b32d-ad92350f4f7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:06:47.984832 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac02667-5053-4698-b32d-ad92350f4f7f service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:06:47.994355 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1050/2110] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:06:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:06:48.264365 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fb8cb2-bb5e-4500-a710-f24a59ac361d None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:06:48.266690 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e9fb8cb2-bb5e-4500-a710-f24a59ac361d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:06:48.279431 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fb8cb2-bb5e-4500-a710-f24a59ac361d None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:06:48.285034 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e945ff-6f9c-4272-ae92-a1117b682edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:06:48.285732 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e945ff-6f9c-4272-ae92-a1117b682edc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:06:48.286014 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e945ff-6f9c-4272-ae92-a1117b682edc admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:06:48.286287 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e945ff-6f9c-4272-ae92-a1117b682edc admin admin] PATH_INFO: `/v3/projects/acc7aac3a9ba4795ab6885d94f7d6cef` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:06:48.287418 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e945ff-6f9c-4272-ae92-a1117b682edc admin admin] RBAC: Authorizing `identity:delete_project(project_id=acc7aac3a9ba4795ab6885d94f7d6cef)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:06:48.294068 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e945ff-6f9c-4272-ae92-a1117b682edc admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:06:48.345703 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-09e945ff-6f9c-4272-ae92-a1117b682edc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'acc7aac3a9ba4795ab6885d94f7d6cef', 'request_id': 'req-09e945ff-6f9c-4272-ae92-a1117b682edc'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:06:48.351098 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-09e945ff-6f9c-4272-ae92-a1117b682edc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'acc7aac3a9ba4795ab6885d94f7d6cef', 'request_id': 'req-09e945ff-6f9c-4272-ae92-a1117b682edc'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:06:48.357315 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-09e945ff-6f9c-4272-ae92-a1117b682edc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'acc7aac3a9ba4795ab6885d94f7d6cef', 'request_id': 'req-09e945ff-6f9c-4272-ae92-a1117b682edc'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:06:48.369494 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-09e945ff-6f9c-4272-ae92-a1117b682edc admin admin] The token cache is being invalidate because project acc7aac3a9ba4795ab6885d94f7d6cef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 04:06:48.369776 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-09e945ff-6f9c-4272-ae92-a1117b682edc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:06:48.370390 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-09e945ff-6f9c-4272-ae92-a1117b682edc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:06:48.371863 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1061/2111] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 04:06:48 2022] DELETE /identity/v3/projects/acc7aac3a9ba4795ab6885d94f7d6cef => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 04:06:48.376304 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac02667-5053-4698-b32d-ad92350f4f7f service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:06:48.385698 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac02667-5053-4698-b32d-ad92350f4f7f service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:06:48.441100 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135869d8-bff8-4132-a526-70e1b0c279e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:06:48.441726 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135869d8-bff8-4132-a526-70e1b0c279e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:06:48.442007 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135869d8-bff8-4132-a526-70e1b0c279e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:06:48.442282 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135869d8-bff8-4132-a526-70e1b0c279e0 admin admin] PATH_INFO: `/v3/users/5bc48f1451c64aff97cba01ca7b00568` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:06:48.443384 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135869d8-bff8-4132-a526-70e1b0c279e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bc48f1451c64aff97cba01ca7b00568)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:06:48.445317 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135869d8-bff8-4132-a526-70e1b0c279e0 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:06:48.490783 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-135869d8-bff8-4132-a526-70e1b0c279e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bc48f1451c64aff97cba01ca7b00568'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:06:48.497827 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-135869d8-bff8-4132-a526-70e1b0c279e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bc48f1451c64aff97cba01ca7b00568'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:06:48.498948 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-135869d8-bff8-4132-a526-70e1b0c279e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bc48f1451c64aff97cba01ca7b00568'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:06:48.510955 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-135869d8-bff8-4132-a526-70e1b0c279e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bc48f1451c64aff97cba01ca7b00568'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:06:48.515613 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:06:48.520143 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-135869d8-bff8-4132-a526-70e1b0c279e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bc48f1451c64aff97cba01ca7b00568'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:06:48.528462 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1051/2112] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 04:06:48 2022] DELETE /identity/v3/users/5bc48f1451c64aff97cba01ca7b00568 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:06:48.803058 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e945ff-6f9c-4272-ae92-a1117b682edc admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:06:48.804529 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-09e945ff-6f9c-4272-ae92-a1117b682edc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:06:48.819322 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e945ff-6f9c-4272-ae92-a1117b682edc admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:06:48.888185 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22a7c6b-1789-4928-b715-f48ae2147abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:06:48.888747 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a7c6b-1789-4928-b715-f48ae2147abb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:06:48.889022 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a7c6b-1789-4928-b715-f48ae2147abb admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:06:48.889292 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22a7c6b-1789-4928-b715-f48ae2147abb admin admin] PATH_INFO: `/v3/projects/33149bbc2fba427d92215279e0d5e5e7` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:06:48.891005 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22a7c6b-1789-4928-b715-f48ae2147abb admin admin] RBAC: Authorizing `identity:delete_project(project_id=33149bbc2fba427d92215279e0d5e5e7)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:06:48.892911 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22a7c6b-1789-4928-b715-f48ae2147abb admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:06:48.937933 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e22a7c6b-1789-4928-b715-f48ae2147abb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33149bbc2fba427d92215279e0d5e5e7', 'request_id': 'req-e22a7c6b-1789-4928-b715-f48ae2147abb'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:06:48.943532 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e22a7c6b-1789-4928-b715-f48ae2147abb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33149bbc2fba427d92215279e0d5e5e7', 'request_id': 'req-e22a7c6b-1789-4928-b715-f48ae2147abb'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:06:48.949570 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e22a7c6b-1789-4928-b715-f48ae2147abb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33149bbc2fba427d92215279e0d5e5e7', 'request_id': 'req-e22a7c6b-1789-4928-b715-f48ae2147abb'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:06:48.961947 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e22a7c6b-1789-4928-b715-f48ae2147abb admin admin] The token cache is being invalidate because project 33149bbc2fba427d92215279e0d5e5e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 04:06:48.962238 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e22a7c6b-1789-4928-b715-f48ae2147abb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:06:48.962860 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e22a7c6b-1789-4928-b715-f48ae2147abb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:06:48.964377 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1062/2113] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 04:06:48 2022] DELETE /identity/v3/projects/33149bbc2fba427d92215279e0d5e5e7 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:06:48.969779 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6416e02e-3879-4906-a452-515563f126b7 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:06:48.970094 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6416e02e-3879-4906-a452-515563f126b7 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:06:48.970369 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6416e02e-3879-4906-a452-515563f126b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:06:49.018598 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-6416e02e-3879-4906-a452-515563f126b7 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:06:49.019778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6416e02e-3879-4906-a452-515563f126b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:06:49.076989 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1052/2114] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:06:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:06:49.086698 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384b4357-59e7-47ad-aac3-f77ef4604a73 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:06:49.086988 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384b4357-59e7-47ad-aac3-f77ef4604a73 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:06:49.087264 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384b4357-59e7-47ad-aac3-f77ef4604a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:06:49.102024 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:06:49.108188 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-384b4357-59e7-47ad-aac3-f77ef4604a73 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:06:49.109950 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-384b4357-59e7-47ad-aac3-f77ef4604a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:06:49.117980 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1063/2115] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:06:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:06:49.122195 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6416e02e-3879-4906-a452-515563f126b7 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:06:49.132574 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6416e02e-3879-4906-a452-515563f126b7 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:06:49.137071 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7572599-c73e-4371-85f3-adb4bf67fd3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:06:49.137665 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7572599-c73e-4371-85f3-adb4bf67fd3b admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:06:49.137944 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7572599-c73e-4371-85f3-adb4bf67fd3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:06:49.138263 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7572599-c73e-4371-85f3-adb4bf67fd3b admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:06:49.138850 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7572599-c73e-4371-85f3-adb4bf67fd3b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 04:06:49.139178 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7572599-c73e-4371-85f3-adb4bf67fd3b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:06:49.141363 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7572599-c73e-4371-85f3-adb4bf67fd3b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:06:49.152625 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1053/2116] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 04:06:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 04:06:49.156730 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384b4357-59e7-47ad-aac3-f77ef4604a73 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:06:49.165864 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384b4357-59e7-47ad-aac3-f77ef4604a73 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:06:49.170481 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2eddce-6f73-4708-8be3-99e99999b295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:06:49.171069 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2eddce-6f73-4708-8be3-99e99999b295 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:06:49.171347 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2eddce-6f73-4708-8be3-99e99999b295 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:06:49.171618 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2eddce-6f73-4708-8be3-99e99999b295 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:06:49.172298 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2eddce-6f73-4708-8be3-99e99999b295 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:06:49.174176 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2eddce-6f73-4708-8be3-99e99999b295 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:06:49.204446 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1064/2117] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 04:06:49 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:06:49.208469 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7572599-c73e-4371-85f3-adb4bf67fd3b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:06:49.218712 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7572599-c73e-4371-85f3-adb4bf67fd3b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:06:49.274965 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b88365a-ac87-416d-85d0-b175e0151d8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:06:49.275677 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b88365a-ac87-416d-85d0-b175e0151d8c admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:06:49.275974 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b88365a-ac87-416d-85d0-b175e0151d8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:06:49.276262 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b88365a-ac87-416d-85d0-b175e0151d8c admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:06:49.276900 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b88365a-ac87-416d-85d0-b175e0151d8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:06:49.278805 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b88365a-ac87-416d-85d0-b175e0151d8c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:06:49.300069 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1054/2118] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 04:06:49 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:06:49.304175 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2eddce-6f73-4708-8be3-99e99999b295 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:06:49.315561 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2eddce-6f73-4708-8be3-99e99999b295 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:06:49.320406 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b16cdb-1379-43d0-8695-97b7c4000081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:06:49.321035 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b16cdb-1379-43d0-8695-97b7c4000081 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:06:49.321308 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b16cdb-1379-43d0-8695-97b7c4000081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:06:49.321577 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b16cdb-1379-43d0-8695-97b7c4000081 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:06:49.322243 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b16cdb-1379-43d0-8695-97b7c4000081 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:06:49.323873 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b16cdb-1379-43d0-8695-97b7c4000081 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:06:49.334215 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1065/2119] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:06:49.338219 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b88365a-ac87-416d-85d0-b175e0151d8c admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:06:49.348200 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b88365a-ac87-416d-85d0-b175e0151d8c admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:06:49.352796 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874c57e0-ac34-4e80-9fc6-6f97ee2110a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:06:49.353391 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c57e0-ac34-4e80-9fc6-6f97ee2110a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:06:49.353665 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c57e0-ac34-4e80-9fc6-6f97ee2110a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:06:49.353934 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874c57e0-ac34-4e80-9fc6-6f97ee2110a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:06:49.354569 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874c57e0-ac34-4e80-9fc6-6f97ee2110a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:06:49.356169 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874c57e0-ac34-4e80-9fc6-6f97ee2110a5 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:06:49.367186 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1055/2120] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:06:49.371215 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b16cdb-1379-43d0-8695-97b7c4000081 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:06:49.382153 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b16cdb-1379-43d0-8695-97b7c4000081 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:06:49.386804 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c61649-a7b6-4968-9a53-1eeb4747a037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:06:49.387447 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c61649-a7b6-4968-9a53-1eeb4747a037 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:06:49.387726 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c61649-a7b6-4968-9a53-1eeb4747a037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:06:49.388031 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c61649-a7b6-4968-9a53-1eeb4747a037 admin admin] PATH_INFO: `/v3/projects/832913a16c2643fd956bd32192fbf988/users/d96c0074dee8489e9da3216ffe40fd91/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:06:49.402351 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c61649-a7b6-4968-9a53-1eeb4747a037 admin admin] RBAC: Authorizing `identity:create_grant(project_id=832913a16c2643fd956bd32192fbf988, user_id=d96c0074dee8489e9da3216ffe40fd91, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:06:49.404971 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c61649-a7b6-4968-9a53-1eeb4747a037 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:06:49.418199 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1066/2121] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 04:06:49 2022] PUT /identity/v3/projects/832913a16c2643fd956bd32192fbf988/users/d96c0074dee8489e9da3216ffe40fd91/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 04:06:49.481814 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f85b0-ba36-47fa-8363-5d43ee9614cc None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:06:49.482390 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f85b0-ba36-47fa-8363-5d43ee9614cc None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:06:49.482390 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80f85b0-ba36-47fa-8363-5d43ee9614cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:06:49.509970 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-c80f85b0-ba36-47fa-8363-5d43ee9614cc None None] MFA Rules not processed for user `d96c0074dee8489e9da3216ffe40fd91`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:06:49.511176 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c80f85b0-ba36-47fa-8363-5d43ee9614cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:06:49.538455 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:06:49.578130 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1056/2122] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:06:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:06:49.582628 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c61649-a7b6-4968-9a53-1eeb4747a037 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:06:49.644215 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c61649-a7b6-4968-9a53-1eeb4747a037 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:06:49.648861 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843b89ea-7224-4cd9-9959-fe6b2c83920f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:06:49.649465 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b89ea-7224-4cd9-9959-fe6b2c83920f admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:06:49.649747 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b89ea-7224-4cd9-9959-fe6b2c83920f admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:06:49.650045 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843b89ea-7224-4cd9-9959-fe6b2c83920f admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:06:49.650647 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843b89ea-7224-4cd9-9959-fe6b2c83920f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:06:49.652672 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843b89ea-7224-4cd9-9959-fe6b2c83920f admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:06:49.678525 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1067/2123] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 04:06:49 2022] POST /identity/v3/projects => generated 387 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:06:49.682776 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80f85b0-ba36-47fa-8363-5d43ee9614cc None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:06:49.693968 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80f85b0-ba36-47fa-8363-5d43ee9614cc None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:06:49.750116 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d970e043-c7ca-4562-84f2-47f502402370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:06:49.750720 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d970e043-c7ca-4562-84f2-47f502402370 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:06:49.750998 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d970e043-c7ca-4562-84f2-47f502402370 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:06:49.751268 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d970e043-c7ca-4562-84f2-47f502402370 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:06:49.751902 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d970e043-c7ca-4562-84f2-47f502402370 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:06:49.753819 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d970e043-c7ca-4562-84f2-47f502402370 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:06:49.774798 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1057/2124] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 04:06:49 2022] POST /identity/v3/users => generated 300 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:06:49.778770 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843b89ea-7224-4cd9-9959-fe6b2c83920f admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:06:49.789036 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843b89ea-7224-4cd9-9959-fe6b2c83920f admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:06:49.793674 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b2d121-4b9b-435e-9f4d-dc067fd90da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:06:49.794273 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b2d121-4b9b-435e-9f4d-dc067fd90da7 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:06:49.794551 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b2d121-4b9b-435e-9f4d-dc067fd90da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:06:49.794820 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b2d121-4b9b-435e-9f4d-dc067fd90da7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:06:49.795454 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b2d121-4b9b-435e-9f4d-dc067fd90da7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:06:49.797171 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b2d121-4b9b-435e-9f4d-dc067fd90da7 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:06:49.807461 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1068/2125] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:06:49.811257 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d970e043-c7ca-4562-84f2-47f502402370 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:06:49.821604 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d970e043-c7ca-4562-84f2-47f502402370 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:06:49.826163 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10daca56-8f10-4ad5-88ad-c94b26b4da1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:06:49.826798 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10daca56-8f10-4ad5-88ad-c94b26b4da1b admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:06:49.827073 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10daca56-8f10-4ad5-88ad-c94b26b4da1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:06:49.827343 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10daca56-8f10-4ad5-88ad-c94b26b4da1b admin admin] PATH_INFO: `/v3/projects/546029c2eb4340b8bb5e8ff5dc79ad0d/users/862acb61dc344dcb83c0073c4ab40e63/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:06:49.842857 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10daca56-8f10-4ad5-88ad-c94b26b4da1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=546029c2eb4340b8bb5e8ff5dc79ad0d, user_id=862acb61dc344dcb83c0073c4ab40e63, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:06:49.845542 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10daca56-8f10-4ad5-88ad-c94b26b4da1b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:06:49.858291 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1058/2126] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 04:06:49 2022] PUT /identity/v3/projects/546029c2eb4340b8bb5e8ff5dc79ad0d/users/862acb61dc344dcb83c0073c4ab40e63/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:06:49.862083 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b2d121-4b9b-435e-9f4d-dc067fd90da7 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:06:49.872230 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b2d121-4b9b-435e-9f4d-dc067fd90da7 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:06:49.933829 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f2e12a-dad6-47bb-ba82-06d7d826b3c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:06:49.934366 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f2e12a-dad6-47bb-ba82-06d7d826b3c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:06:49.934638 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f2e12a-dad6-47bb-ba82-06d7d826b3c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:06:49.934911 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f2e12a-dad6-47bb-ba82-06d7d826b3c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:06:49.935538 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f2e12a-dad6-47bb-ba82-06d7d826b3c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:06:49.937713 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f2e12a-dad6-47bb-ba82-06d7d826b3c0 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:06:49.947212 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1069/2127] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:06:49.951321 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10daca56-8f10-4ad5-88ad-c94b26b4da1b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:06:49.962257 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10daca56-8f10-4ad5-88ad-c94b26b4da1b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:06:49.966875 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb5a181-0610-42a9-be86-a64621490d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:06:49.967521 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5a181-0610-42a9-be86-a64621490d5d admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:06:49.967804 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5a181-0610-42a9-be86-a64621490d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:06:49.968118 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb5a181-0610-42a9-be86-a64621490d5d admin admin] PATH_INFO: `/v3/projects/546029c2eb4340b8bb5e8ff5dc79ad0d/users/862acb61dc344dcb83c0073c4ab40e63/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:06:49.970063 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb5a181-0610-42a9-be86-a64621490d5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=546029c2eb4340b8bb5e8ff5dc79ad0d, user_id=862acb61dc344dcb83c0073c4ab40e63, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:06:49.972749 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb5a181-0610-42a9-be86-a64621490d5d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:06:49.982867 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1059/2128] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 04:06:49 2022] PUT /identity/v3/projects/546029c2eb4340b8bb5e8ff5dc79ad0d/users/862acb61dc344dcb83c0073c4ab40e63/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:06:49.992855 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f54e7-e29e-4685-8072-93df0366d174 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:06:49.993150 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f54e7-e29e-4685-8072-93df0366d174 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:06:49.993427 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968f54e7-e29e-4685-8072-93df0366d174 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:06:50.018981 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:06:50.022325 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-968f54e7-e29e-4685-8072-93df0366d174 None None] MFA Rules not processed for user `862acb61dc344dcb83c0073c4ab40e63`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:06:50.024116 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-968f54e7-e29e-4685-8072-93df0366d174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:06:50.091972 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1070/2129] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:06:49 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:06:50.103024 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb5a181-0610-42a9-be86-a64621490d5d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:06:50.104722 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3cb5a181-0610-42a9-be86-a64621490d5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:06:50.162268 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb5a181-0610-42a9-be86-a64621490d5d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:06:50.165905 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc567a0b-5df1-4337-afe9-7cd69ed4f5a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:06:50.166504 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc567a0b-5df1-4337-afe9-7cd69ed4f5a9 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:06:50.166777 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc567a0b-5df1-4337-afe9-7cd69ed4f5a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:06:50.167045 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc567a0b-5df1-4337-afe9-7cd69ed4f5a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:06:50.176554 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc567a0b-5df1-4337-afe9-7cd69ed4f5a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:06:50.177997 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc567a0b-5df1-4337-afe9-7cd69ed4f5a9 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:06:50.187877 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1060/2130] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:06:50 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:06:50.253047 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968f54e7-e29e-4685-8072-93df0366d174 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:06:50.313958 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968f54e7-e29e-4685-8072-93df0366d174 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:06:50.318634 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef1f563-5226-40cd-a777-3d532504ca55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:06:50.319260 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef1f563-5226-40cd-a777-3d532504ca55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:06:50.319536 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef1f563-5226-40cd-a777-3d532504ca55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:06:50.319831 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef1f563-5226-40cd-a777-3d532504ca55 admin admin] PATH_INFO: `/v3/users/d96c0074dee8489e9da3216ffe40fd91` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:06:50.320979 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef1f563-5226-40cd-a777-3d532504ca55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d96c0074dee8489e9da3216ffe40fd91)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:06:50.322908 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef1f563-5226-40cd-a777-3d532504ca55 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:06:50.369405 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2ef1f563-5226-40cd-a777-3d532504ca55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd96c0074dee8489e9da3216ffe40fd91'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:06:50.370143 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2ef1f563-5226-40cd-a777-3d532504ca55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd96c0074dee8489e9da3216ffe40fd91'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:06:50.370768 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2ef1f563-5226-40cd-a777-3d532504ca55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd96c0074dee8489e9da3216ffe40fd91'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:06:50.383031 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2ef1f563-5226-40cd-a777-3d532504ca55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd96c0074dee8489e9da3216ffe40fd91'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:06:50.390204 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2ef1f563-5226-40cd-a777-3d532504ca55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd96c0074dee8489e9da3216ffe40fd91'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:06:50.402230 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1071/2131] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 04:06:50 2022] DELETE /identity/v3/users/d96c0074dee8489e9da3216ffe40fd91 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:06:50.407526 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c9fe43-c020-46d8-924b-58752252518f None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:06:50.407806 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c9fe43-c020-46d8-924b-58752252518f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:06:50.408109 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c9fe43-c020-46d8-924b-58752252518f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:06:50.427291 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a1c9fe43-c020-46d8-924b-58752252518f None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:06:50.428508 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a1c9fe43-c020-46d8-924b-58752252518f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:06:50.444599 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:06:50.488265 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1061/2132] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:06:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:06:50.498062 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef1f563-5226-40cd-a777-3d532504ca55 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:06:50.499740 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-2ef1f563-5226-40cd-a777-3d532504ca55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:06:50.519352 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:06:50.559849 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef1f563-5226-40cd-a777-3d532504ca55 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:06:50.563485 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7f2ab9-871a-426b-9e5e-c3ec6a516c45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:06:50.564115 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7f2ab9-871a-426b-9e5e-c3ec6a516c45 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:06:50.564394 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7f2ab9-871a-426b-9e5e-c3ec6a516c45 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:06:50.564661 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7f2ab9-871a-426b-9e5e-c3ec6a516c45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:06:50.573533 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7f2ab9-871a-426b-9e5e-c3ec6a516c45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:06:50.574943 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da7f2ab9-871a-426b-9e5e-c3ec6a516c45 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:06:50.584459 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1072/2133] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:06:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:06:50.854791 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c9fe43-c020-46d8-924b-58752252518f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:06:50.856438 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a1c9fe43-c020-46d8-924b-58752252518f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:06:50.879418 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c9fe43-c020-46d8-924b-58752252518f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:06:50.884462 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ecc6ca-b02c-4f72-982f-3e14b5aadce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:06:50.885209 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ecc6ca-b02c-4f72-982f-3e14b5aadce3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:06:50.885380 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ecc6ca-b02c-4f72-982f-3e14b5aadce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:06:50.885658 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ecc6ca-b02c-4f72-982f-3e14b5aadce3 admin admin] PATH_INFO: `/v3/projects/832913a16c2643fd956bd32192fbf988` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:06:50.886910 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ecc6ca-b02c-4f72-982f-3e14b5aadce3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=832913a16c2643fd956bd32192fbf988)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:06:50.889216 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ecc6ca-b02c-4f72-982f-3e14b5aadce3 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:06:50.962472 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e1ecc6ca-b02c-4f72-982f-3e14b5aadce3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '832913a16c2643fd956bd32192fbf988', 'request_id': 'req-e1ecc6ca-b02c-4f72-982f-3e14b5aadce3'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:06:50.969693 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e1ecc6ca-b02c-4f72-982f-3e14b5aadce3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '832913a16c2643fd956bd32192fbf988', 'request_id': 'req-e1ecc6ca-b02c-4f72-982f-3e14b5aadce3'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:06:50.980369 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e1ecc6ca-b02c-4f72-982f-3e14b5aadce3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '832913a16c2643fd956bd32192fbf988', 'request_id': 'req-e1ecc6ca-b02c-4f72-982f-3e14b5aadce3'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:06:50.997554 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e1ecc6ca-b02c-4f72-982f-3e14b5aadce3 admin admin] The token cache is being invalidate because project 832913a16c2643fd956bd32192fbf988 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 04:06:50.998980 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e1ecc6ca-b02c-4f72-982f-3e14b5aadce3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:06:51.001446 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e1ecc6ca-b02c-4f72-982f-3e14b5aadce3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:06:51.003825 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1062/2134] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 04:06:50 2022] DELETE /identity/v3/projects/832913a16c2643fd956bd32192fbf988 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:06:51.007696 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7f2ab9-871a-426b-9e5e-c3ec6a516c45 service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:06:51.019413 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7f2ab9-871a-426b-9e5e-c3ec6a516c45 service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:06:51.120064 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14acb894-8735-41c1-9508-ffc23d72f909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:06:51.121142 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14acb894-8735-41c1-9508-ffc23d72f909 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:06:51.121429 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14acb894-8735-41c1-9508-ffc23d72f909 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:06:51.121701 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14acb894-8735-41c1-9508-ffc23d72f909 admin admin] PATH_INFO: `/v3/users/862acb61dc344dcb83c0073c4ab40e63` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:06:51.122899 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14acb894-8735-41c1-9508-ffc23d72f909 admin admin] RBAC: Authorizing `identity:delete_user(user_id=862acb61dc344dcb83c0073c4ab40e63)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:06:51.126211 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14acb894-8735-41c1-9508-ffc23d72f909 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:06:51.198628 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-14acb894-8735-41c1-9508-ffc23d72f909 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '862acb61dc344dcb83c0073c4ab40e63'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:06:51.199612 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-14acb894-8735-41c1-9508-ffc23d72f909 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '862acb61dc344dcb83c0073c4ab40e63'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:06:51.200415 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-14acb894-8735-41c1-9508-ffc23d72f909 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '862acb61dc344dcb83c0073c4ab40e63'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:06:51.214823 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-14acb894-8735-41c1-9508-ffc23d72f909 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '862acb61dc344dcb83c0073c4ab40e63'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:06:51.222924 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-14acb894-8735-41c1-9508-ffc23d72f909 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '862acb61dc344dcb83c0073c4ab40e63'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:06:51.236210 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1073/2135] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 04:06:51 2022] DELETE /identity/v3/users/862acb61dc344dcb83c0073c4ab40e63 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:06:51.604272 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ecc6ca-b02c-4f72-982f-3e14b5aadce3 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:06:51.605813 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e1ecc6ca-b02c-4f72-982f-3e14b5aadce3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:06:51.619664 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ecc6ca-b02c-4f72-982f-3e14b5aadce3 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:06:51.678429 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6587b89-3f4a-41c4-8856-2eed9afe09cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:06:51.679093 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6587b89-3f4a-41c4-8856-2eed9afe09cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:06:51.679403 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6587b89-3f4a-41c4-8856-2eed9afe09cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:06:51.679741 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6587b89-3f4a-41c4-8856-2eed9afe09cc admin admin] PATH_INFO: `/v3/projects/546029c2eb4340b8bb5e8ff5dc79ad0d` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:06:51.680813 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6587b89-3f4a-41c4-8856-2eed9afe09cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=546029c2eb4340b8bb5e8ff5dc79ad0d)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:06:51.682699 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6587b89-3f4a-41c4-8856-2eed9afe09cc admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:06:51.727766 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:06:51.729025 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e6587b89-3f4a-41c4-8856-2eed9afe09cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '546029c2eb4340b8bb5e8ff5dc79ad0d', 'request_id': 'req-e6587b89-3f4a-41c4-8856-2eed9afe09cc'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:06:51.737373 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e6587b89-3f4a-41c4-8856-2eed9afe09cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '546029c2eb4340b8bb5e8ff5dc79ad0d', 'request_id': 'req-e6587b89-3f4a-41c4-8856-2eed9afe09cc'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:06:51.744762 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e6587b89-3f4a-41c4-8856-2eed9afe09cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '546029c2eb4340b8bb5e8ff5dc79ad0d', 'request_id': 'req-e6587b89-3f4a-41c4-8856-2eed9afe09cc'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:06:51.755717 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e6587b89-3f4a-41c4-8856-2eed9afe09cc admin admin] The token cache is being invalidate because project 546029c2eb4340b8bb5e8ff5dc79ad0d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 04:06:51.756033 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e6587b89-3f4a-41c4-8856-2eed9afe09cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:06:51.756663 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e6587b89-3f4a-41c4-8856-2eed9afe09cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:06:51.758155 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1063/2136] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 04:06:51 2022] DELETE /identity/v3/projects/546029c2eb4340b8bb5e8ff5dc79ad0d => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 04:06:51.763290 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b73e12-d1f7-47d9-b7b5-4fa6ad82c188 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:06:51.763804 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b73e12-d1f7-47d9-b7b5-4fa6ad82c188 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:06:51.763911 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b73e12-d1f7-47d9-b7b5-4fa6ad82c188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:06:51.784463 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-b9b73e12-d1f7-47d9-b7b5-4fa6ad82c188 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:06:51.785666 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-b9b73e12-d1f7-47d9-b7b5-4fa6ad82c188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:06:51.818716 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:06:51.845884 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1074/2137] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:06:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:06:51.854624 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00281df-9c69-4108-9e2d-ba8a9f4ab7b0 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:06:51.854980 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00281df-9c69-4108-9e2d-ba8a9f4ab7b0 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:06:51.855264 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00281df-9c69-4108-9e2d-ba8a9f4ab7b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:06:51.875602 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d00281df-9c69-4108-9e2d-ba8a9f4ab7b0 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:06:51.876827 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d00281df-9c69-4108-9e2d-ba8a9f4ab7b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:06:51.884624 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1064/2138] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:06:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:06:51.888904 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b73e12-d1f7-47d9-b7b5-4fa6ad82c188 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:06:51.898427 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b73e12-d1f7-47d9-b7b5-4fa6ad82c188 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:06:51.902948 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55309fbe-de38-43c0-8f37-ddfbac0200bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:06:51.903544 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55309fbe-de38-43c0-8f37-ddfbac0200bc admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:06:51.903823 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55309fbe-de38-43c0-8f37-ddfbac0200bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:06:51.904152 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55309fbe-de38-43c0-8f37-ddfbac0200bc admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:06:51.904792 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55309fbe-de38-43c0-8f37-ddfbac0200bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 04:06:51.905090 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55309fbe-de38-43c0-8f37-ddfbac0200bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:06:51.907214 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55309fbe-de38-43c0-8f37-ddfbac0200bc admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:06:51.918518 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1075/2139] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 04:06:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 04:06:51.923349 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00281df-9c69-4108-9e2d-ba8a9f4ab7b0 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:06:51.932618 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00281df-9c69-4108-9e2d-ba8a9f4ab7b0 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:06:51.937392 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4fd139-82de-4427-82a5-24eae22ed209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:06:51.937992 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4fd139-82de-4427-82a5-24eae22ed209 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:06:51.938702 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4fd139-82de-4427-82a5-24eae22ed209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:06:51.938702 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4fd139-82de-4427-82a5-24eae22ed209 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:06:51.939159 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4fd139-82de-4427-82a5-24eae22ed209 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:06:51.941080 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f4fd139-82de-4427-82a5-24eae22ed209 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:06:51.976968 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1065/2140] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 04:06:51 2022] POST /identity/v3/projects => generated 387 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:06:51.977936 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55309fbe-de38-43c0-8f37-ddfbac0200bc admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:06:51.988816 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55309fbe-de38-43c0-8f37-ddfbac0200bc admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:06:52.046061 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a498342-4ae4-45b5-b3d4-074c9d1fa80e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:06:52.047277 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a498342-4ae4-45b5-b3d4-074c9d1fa80e admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:06:52.047558 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a498342-4ae4-45b5-b3d4-074c9d1fa80e admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:06:52.047827 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a498342-4ae4-45b5-b3d4-074c9d1fa80e admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:06:52.048482 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a498342-4ae4-45b5-b3d4-074c9d1fa80e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:06:52.050367 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a498342-4ae4-45b5-b3d4-074c9d1fa80e admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:06:52.071466 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1076/2141] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 04:06:51 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:06:52.075691 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4fd139-82de-4427-82a5-24eae22ed209 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:06:52.085943 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4fd139-82de-4427-82a5-24eae22ed209 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:06:52.090627 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeabb3f-36ec-4f69-9de2-634fd3f930b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:06:52.091308 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeabb3f-36ec-4f69-9de2-634fd3f930b3 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:06:52.091745 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeabb3f-36ec-4f69-9de2-634fd3f930b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:06:52.092120 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eeabb3f-36ec-4f69-9de2-634fd3f930b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:06:52.092755 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeabb3f-36ec-4f69-9de2-634fd3f930b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:06:52.094906 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eeabb3f-36ec-4f69-9de2-634fd3f930b3 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:06:52.104792 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1066/2142] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:06:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:06:52.109317 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a498342-4ae4-45b5-b3d4-074c9d1fa80e admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:06:52.120480 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a498342-4ae4-45b5-b3d4-074c9d1fa80e admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:06:52.125835 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27807a04-a63b-4d29-85db-c5d3b08fb095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:06:52.126872 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27807a04-a63b-4d29-85db-c5d3b08fb095 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:06:52.127121 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27807a04-a63b-4d29-85db-c5d3b08fb095 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:06:52.127390 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27807a04-a63b-4d29-85db-c5d3b08fb095 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:06:52.128067 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27807a04-a63b-4d29-85db-c5d3b08fb095 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:06:52.129768 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27807a04-a63b-4d29-85db-c5d3b08fb095 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:06:52.140908 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1077/2143] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:06:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 04:06:52.144853 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeabb3f-36ec-4f69-9de2-634fd3f930b3 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:06:52.156921 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eeabb3f-36ec-4f69-9de2-634fd3f930b3 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:06:52.161762 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6541d5c-710f-4991-b349-7153a6603c77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:06:52.162510 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6541d5c-710f-4991-b349-7153a6603c77 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:06:52.162852 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6541d5c-710f-4991-b349-7153a6603c77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:06:52.163190 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6541d5c-710f-4991-b349-7153a6603c77 admin admin] PATH_INFO: `/v3/projects/194eed487cdf4563835e13272e6f5c8a/users/88036b5c84c34f9992fceac977dc3027/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:06:52.180898 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6541d5c-710f-4991-b349-7153a6603c77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=194eed487cdf4563835e13272e6f5c8a, user_id=88036b5c84c34f9992fceac977dc3027, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:06:52.184487 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6541d5c-710f-4991-b349-7153a6603c77 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:06:52.198557 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1067/2144] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 04:06:52 2022] PUT /identity/v3/projects/194eed487cdf4563835e13272e6f5c8a/users/88036b5c84c34f9992fceac977dc3027/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:06:52.209921 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8e78f4-628d-4831-b610-c697e9368943 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:06:52.210250 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8e78f4-628d-4831-b610-c697e9368943 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:06:52.210566 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8e78f4-628d-4831-b610-c697e9368943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:06:52.240954 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-9d8e78f4-628d-4831-b610-c697e9368943 None None] MFA Rules not processed for user `88036b5c84c34f9992fceac977dc3027`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:06:52.242398 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-9d8e78f4-628d-4831-b610-c697e9368943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:06:52.320189 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1078/2145] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:06:52 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 04:06:52.324826 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6541d5c-710f-4991-b349-7153a6603c77 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:06:52.403704 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6541d5c-710f-4991-b349-7153a6603c77 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:06:52.409061 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c58645-8ef1-4884-b01f-d45597a9eea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:06:52.409665 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c58645-8ef1-4884-b01f-d45597a9eea3 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:06:52.409947 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c58645-8ef1-4884-b01f-d45597a9eea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:06:52.410223 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c58645-8ef1-4884-b01f-d45597a9eea3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:06:52.410855 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c58645-8ef1-4884-b01f-d45597a9eea3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:06:52.412783 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c58645-8ef1-4884-b01f-d45597a9eea3 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:06:52.438404 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1068/2146] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 04:06:52 2022] POST /identity/v3/projects => generated 387 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:06:52.442532 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8e78f4-628d-4831-b610-c697e9368943 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:06:52.453888 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8e78f4-628d-4831-b610-c697e9368943 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:06:52.486095 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:06:52.511709 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0bbbbe-00a5-48a0-b595-a4c9b191e13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:06:52.512287 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0bbbbe-00a5-48a0-b595-a4c9b191e13f admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:06:52.512572 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0bbbbe-00a5-48a0-b595-a4c9b191e13f admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:06:52.512848 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0bbbbe-00a5-48a0-b595-a4c9b191e13f admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:06:52.513477 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0bbbbe-00a5-48a0-b595-a4c9b191e13f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:06:52.515935 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0bbbbe-00a5-48a0-b595-a4c9b191e13f admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:06:52.538866 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1079/2147] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 04:06:52 2022] POST /identity/v3/users => generated 294 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:06:52.542961 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c58645-8ef1-4884-b01f-d45597a9eea3 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:06:52.553229 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c58645-8ef1-4884-b01f-d45597a9eea3 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:06:52.557853 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4272c27b-8d87-4da4-89b5-a9762c3e7b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:06:52.558446 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4272c27b-8d87-4da4-89b5-a9762c3e7b3c admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:06:52.558729 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4272c27b-8d87-4da4-89b5-a9762c3e7b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:06:52.559003 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4272c27b-8d87-4da4-89b5-a9762c3e7b3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:06:52.559733 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4272c27b-8d87-4da4-89b5-a9762c3e7b3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:06:52.561376 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4272c27b-8d87-4da4-89b5-a9762c3e7b3c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:06:52.572366 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1069/2148] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:06:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:06:52.576190 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0bbbbe-00a5-48a0-b595-a4c9b191e13f admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:06:52.586578 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0bbbbe-00a5-48a0-b595-a4c9b191e13f admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:06:52.591171 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7790fa3-51f2-4706-a0f8-b40337391608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:06:52.591798 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7790fa3-51f2-4706-a0f8-b40337391608 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:06:52.592096 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7790fa3-51f2-4706-a0f8-b40337391608 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:06:52.592368 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7790fa3-51f2-4706-a0f8-b40337391608 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:06:52.593025 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7790fa3-51f2-4706-a0f8-b40337391608 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:06:52.594601 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7790fa3-51f2-4706-a0f8-b40337391608 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:06:52.604464 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1080/2149] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:06:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:06:52.608499 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4272c27b-8d87-4da4-89b5-a9762c3e7b3c admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:06:52.618326 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4272c27b-8d87-4da4-89b5-a9762c3e7b3c admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:06:52.622962 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a73964-f907-4911-9d0a-5da4a8ae73da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:06:52.623862 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a73964-f907-4911-9d0a-5da4a8ae73da admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:06:52.624172 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a73964-f907-4911-9d0a-5da4a8ae73da admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:06:52.624979 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a73964-f907-4911-9d0a-5da4a8ae73da admin admin] PATH_INFO: `/v3/projects/ae6f3b589faa425fbacb0260387651bf/users/6bb007964d4a4aabbed06ad0afb0aae9/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:06:52.639354 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a73964-f907-4911-9d0a-5da4a8ae73da admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae6f3b589faa425fbacb0260387651bf, user_id=6bb007964d4a4aabbed06ad0afb0aae9, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:06:52.642565 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a73964-f907-4911-9d0a-5da4a8ae73da admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:06:52.655211 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1070/2150] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 04:06:52 2022] PUT /identity/v3/projects/ae6f3b589faa425fbacb0260387651bf/users/6bb007964d4a4aabbed06ad0afb0aae9/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:06:52.665619 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd6aed-2878-4ab1-8392-0458e125cc57 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:06:52.665910 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd6aed-2878-4ab1-8392-0458e125cc57 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:06:52.666186 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bd6aed-2878-4ab1-8392-0458e125cc57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:06:52.694341 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-84bd6aed-2878-4ab1-8392-0458e125cc57 None None] MFA Rules not processed for user `6bb007964d4a4aabbed06ad0afb0aae9`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:06:52.695551 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-84bd6aed-2878-4ab1-8392-0458e125cc57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:06:52.766086 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1081/2151] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:06:52 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:06:52.770571 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a73964-f907-4911-9d0a-5da4a8ae73da admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:06:52.795512 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:06:52.834095 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a73964-f907-4911-9d0a-5da4a8ae73da admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:06:52.838727 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba573e-03e2-4049-9cf5-6afdfbf310ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:06:52.839321 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba573e-03e2-4049-9cf5-6afdfbf310ee admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:06:52.839597 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba573e-03e2-4049-9cf5-6afdfbf310ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:06:52.839866 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba573e-03e2-4049-9cf5-6afdfbf310ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:06:52.840511 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ba573e-03e2-4049-9cf5-6afdfbf310ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:06:52.842392 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ba573e-03e2-4049-9cf5-6afdfbf310ee admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:06:52.868317 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1071/2152] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 04:06:52 2022] POST /identity/v3/projects => generated 383 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:06:52.872523 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bd6aed-2878-4ab1-8392-0458e125cc57 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:06:52.883863 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bd6aed-2878-4ab1-8392-0458e125cc57 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:06:52.940735 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24a648b-53e7-4e8a-ade7-ab7a6dbc3e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:06:52.941342 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a648b-53e7-4e8a-ade7-ab7a6dbc3e98 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:06:52.941619 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a648b-53e7-4e8a-ade7-ab7a6dbc3e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:06:52.941887 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24a648b-53e7-4e8a-ade7-ab7a6dbc3e98 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:06:52.942528 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24a648b-53e7-4e8a-ade7-ab7a6dbc3e98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:06:52.944477 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24a648b-53e7-4e8a-ade7-ab7a6dbc3e98 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:06:52.966064 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1082/2153] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 04:06:52 2022] POST /identity/v3/users => generated 298 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:06:52.970008 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba573e-03e2-4049-9cf5-6afdfbf310ee admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:06:52.980026 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba573e-03e2-4049-9cf5-6afdfbf310ee admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:06:52.985351 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f224716d-0c88-47f1-8bc8-9badd4dc2975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:06:52.985859 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f224716d-0c88-47f1-8bc8-9badd4dc2975 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:06:52.986136 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f224716d-0c88-47f1-8bc8-9badd4dc2975 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:06:52.986403 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f224716d-0c88-47f1-8bc8-9badd4dc2975 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:06:52.987103 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f224716d-0c88-47f1-8bc8-9badd4dc2975 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:06:52.988801 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f224716d-0c88-47f1-8bc8-9badd4dc2975 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:06:53.002003 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1072/2154] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:06:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:06:53.006546 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24a648b-53e7-4e8a-ade7-ab7a6dbc3e98 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:06:53.017076 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24a648b-53e7-4e8a-ade7-ab7a6dbc3e98 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:06:53.022082 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b641da-56d2-42c6-8d3c-d91c0102f05d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:06:53.022777 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b641da-56d2-42c6-8d3c-d91c0102f05d admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:06:53.023059 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b641da-56d2-42c6-8d3c-d91c0102f05d admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:06:53.023335 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b641da-56d2-42c6-8d3c-d91c0102f05d admin admin] PATH_INFO: `/v3/projects/b391895835f941a69ebea19b124469b2/users/e50eb73a8048482f9cd2e0185d7b0f29/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:06:53.038223 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b641da-56d2-42c6-8d3c-d91c0102f05d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b391895835f941a69ebea19b124469b2, user_id=e50eb73a8048482f9cd2e0185d7b0f29, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:06:53.040909 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b641da-56d2-42c6-8d3c-d91c0102f05d admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:06:53.054021 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1083/2155] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 04:06:53 2022] PUT /identity/v3/projects/b391895835f941a69ebea19b124469b2/users/e50eb73a8048482f9cd2e0185d7b0f29/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:06:53.057803 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f224716d-0c88-47f1-8bc8-9badd4dc2975 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:06:53.068414 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f224716d-0c88-47f1-8bc8-9badd4dc2975 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:06:53.125711 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79afcb0f-3a41-480c-90af-f788a411a3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:06:53.126312 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79afcb0f-3a41-480c-90af-f788a411a3f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:06:53.126592 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79afcb0f-3a41-480c-90af-f788a411a3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:06:53.126938 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79afcb0f-3a41-480c-90af-f788a411a3f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:06:53.127581 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79afcb0f-3a41-480c-90af-f788a411a3f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:06:53.129198 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79afcb0f-3a41-480c-90af-f788a411a3f8 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:06:53.138709 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1073/2156] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:06:53 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:06:53.143380 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b641da-56d2-42c6-8d3c-d91c0102f05d admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:06:53.153609 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b641da-56d2-42c6-8d3c-d91c0102f05d admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:06:53.158171 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8e5032-f31c-424c-9d15-957db4357d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:06:53.158811 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8e5032-f31c-424c-9d15-957db4357d4d admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:06:53.159088 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8e5032-f31c-424c-9d15-957db4357d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:06:53.159362 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8e5032-f31c-424c-9d15-957db4357d4d admin admin] PATH_INFO: `/v3/projects/b391895835f941a69ebea19b124469b2/users/e50eb73a8048482f9cd2e0185d7b0f29/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:06:53.161299 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8e5032-f31c-424c-9d15-957db4357d4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b391895835f941a69ebea19b124469b2, user_id=e50eb73a8048482f9cd2e0185d7b0f29, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:06:53.163926 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8e5032-f31c-424c-9d15-957db4357d4d admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:06:53.166937 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:06:53.176699 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1084/2157] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 04:06:53 2022] PUT /identity/v3/projects/b391895835f941a69ebea19b124469b2/users/e50eb73a8048482f9cd2e0185d7b0f29/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:06:53.188471 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f02aff4-927c-451e-af9f-531bf870e83a None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:06:53.188757 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f02aff4-927c-451e-af9f-531bf870e83a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:06:53.189031 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f02aff4-927c-451e-af9f-531bf870e83a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:06:53.216209 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-4f02aff4-927c-451e-af9f-531bf870e83a None None] MFA Rules not processed for user `e50eb73a8048482f9cd2e0185d7b0f29`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:06:53.217414 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4f02aff4-927c-451e-af9f-531bf870e83a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:06:53.284846 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1074/2158] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:06:53 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:06:53.295441 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8e5032-f31c-424c-9d15-957db4357d4d admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:06:53.297668 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ee8e5032-f31c-424c-9d15-957db4357d4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:06:53.356935 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8e5032-f31c-424c-9d15-957db4357d4d admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:06:53.361160 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f339191f-942e-47f6-844c-eb7ccd8ad695 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:06:53.361760 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f339191f-942e-47f6-844c-eb7ccd8ad695 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:06:53.362038 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f339191f-942e-47f6-844c-eb7ccd8ad695 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:06:53.362307 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f339191f-942e-47f6-844c-eb7ccd8ad695 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:06:53.371203 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f339191f-942e-47f6-844c-eb7ccd8ad695 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:06:53.372733 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f339191f-942e-47f6-844c-eb7ccd8ad695 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:06:53.383617 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1085/2159] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:06:53 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:06:53.965487 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f02aff4-927c-451e-af9f-531bf870e83a None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:06:53.977087 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f02aff4-927c-451e-af9f-531bf870e83a None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:06:53.982011 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a256a094-6cab-4ed2-9aee-410fb1310097 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:06:53.983420 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a256a094-6cab-4ed2-9aee-410fb1310097 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:06:53.983738 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a256a094-6cab-4ed2-9aee-410fb1310097 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:06:53.984058 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a256a094-6cab-4ed2-9aee-410fb1310097 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:06:53.999198 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:06:54.045269 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a256a094-6cab-4ed2-9aee-410fb1310097 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:06:54.046735 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a256a094-6cab-4ed2-9aee-410fb1310097 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:06:54.055736 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1075/2160] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:06:53 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:07:01.304412 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f339191f-942e-47f6-844c-eb7ccd8ad695 service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:01.306110 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f339191f-942e-47f6-844c-eb7ccd8ad695 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:01.331157 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f339191f-942e-47f6-844c-eb7ccd8ad695 service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:01.334948 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26109e41-273c-4f09-81e8-d62771ec9c9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:01.335556 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26109e41-273c-4f09-81e8-d62771ec9c9f service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:01.335836 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26109e41-273c-4f09-81e8-d62771ec9c9f service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:01.336436 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26109e41-273c-4f09-81e8-d62771ec9c9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:01.402827 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26109e41-273c-4f09-81e8-d62771ec9c9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:01.404304 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26109e41-273c-4f09-81e8-d62771ec9c9f service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:01.424801 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1086/2161] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:07:01 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:07:04.051583 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=20 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:07:05.670227 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a256a094-6cab-4ed2-9aee-410fb1310097 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:05.737319 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a256a094-6cab-4ed2-9aee-410fb1310097 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:05.742155 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05658ee-ee7c-4d57-b1a3-e3101d19bf35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:05.742807 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05658ee-ee7c-4d57-b1a3-e3101d19bf35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:05.743089 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05658ee-ee7c-4d57-b1a3-e3101d19bf35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:05.743366 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05658ee-ee7c-4d57-b1a3-e3101d19bf35 admin admin] PATH_INFO: `/v3/users/88036b5c84c34f9992fceac977dc3027` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:05.744511 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05658ee-ee7c-4d57-b1a3-e3101d19bf35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88036b5c84c34f9992fceac977dc3027)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:05.746431 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05658ee-ee7c-4d57-b1a3-e3101d19bf35 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:05.794896 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a05658ee-ee7c-4d57-b1a3-e3101d19bf35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88036b5c84c34f9992fceac977dc3027'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:05.802103 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a05658ee-ee7c-4d57-b1a3-e3101d19bf35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88036b5c84c34f9992fceac977dc3027'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:05.802812 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a05658ee-ee7c-4d57-b1a3-e3101d19bf35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88036b5c84c34f9992fceac977dc3027'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:05.815227 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a05658ee-ee7c-4d57-b1a3-e3101d19bf35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88036b5c84c34f9992fceac977dc3027'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:05.822738 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a05658ee-ee7c-4d57-b1a3-e3101d19bf35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88036b5c84c34f9992fceac977dc3027'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:05.829546 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1076/2162] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 04:07:05 2022] DELETE /identity/v3/users/88036b5c84c34f9992fceac977dc3027 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:07:05.834116 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c68c964-fab4-43ea-a920-7aed6deac821 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:05.834686 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c68c964-fab4-43ea-a920-7aed6deac821 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:05.834686 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c68c964-fab4-43ea-a920-7aed6deac821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:05.854669 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-9c68c964-fab4-43ea-a920-7aed6deac821 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:07:05.855882 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-9c68c964-fab4-43ea-a920-7aed6deac821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:05.917765 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1087/2163] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:07:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 04:07:05.928947 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05658ee-ee7c-4d57-b1a3-e3101d19bf35 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:05.930458 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a05658ee-ee7c-4d57-b1a3-e3101d19bf35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:05.999581 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05658ee-ee7c-4d57-b1a3-e3101d19bf35 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:06.005052 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226feb25-7953-4ed6-8836-e356f3a6a23b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:06.005968 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226feb25-7953-4ed6-8836-e356f3a6a23b service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:06.006280 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226feb25-7953-4ed6-8836-e356f3a6a23b service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:06.006586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226feb25-7953-4ed6-8836-e356f3a6a23b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:06.022658 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226feb25-7953-4ed6-8836-e356f3a6a23b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:06.024203 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226feb25-7953-4ed6-8836-e356f3a6a23b service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:06.043497 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1077/2164] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:07:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:07:06.470990 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c68c964-fab4-43ea-a920-7aed6deac821 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:06.472479 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-9c68c964-fab4-43ea-a920-7aed6deac821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:06.494357 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c68c964-fab4-43ea-a920-7aed6deac821 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:06.503987 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8651364-6025-4ae0-b224-cab527c331af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:06.504664 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8651364-6025-4ae0-b224-cab527c331af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:06.504955 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8651364-6025-4ae0-b224-cab527c331af admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:06.505230 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8651364-6025-4ae0-b224-cab527c331af admin admin] PATH_INFO: `/v3/projects/194eed487cdf4563835e13272e6f5c8a` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:06.508363 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8651364-6025-4ae0-b224-cab527c331af admin admin] RBAC: Authorizing `identity:delete_project(project_id=194eed487cdf4563835e13272e6f5c8a)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:06.510296 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8651364-6025-4ae0-b224-cab527c331af admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:06.543607 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:07:06.571543 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e8651364-6025-4ae0-b224-cab527c331af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '194eed487cdf4563835e13272e6f5c8a', 'request_id': 'req-e8651364-6025-4ae0-b224-cab527c331af'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:06.578413 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e8651364-6025-4ae0-b224-cab527c331af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '194eed487cdf4563835e13272e6f5c8a', 'request_id': 'req-e8651364-6025-4ae0-b224-cab527c331af'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:06.585359 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e8651364-6025-4ae0-b224-cab527c331af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '194eed487cdf4563835e13272e6f5c8a', 'request_id': 'req-e8651364-6025-4ae0-b224-cab527c331af'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:06.600099 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e8651364-6025-4ae0-b224-cab527c331af admin admin] The token cache is being invalidate because project 194eed487cdf4563835e13272e6f5c8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 04:07:06.600565 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e8651364-6025-4ae0-b224-cab527c331af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:06.602592 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e8651364-6025-4ae0-b224-cab527c331af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:06.607744 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1088/2165] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 04:07:06 2022] DELETE /identity/v3/projects/194eed487cdf4563835e13272e6f5c8a => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:07:06.608940 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226feb25-7953-4ed6-8836-e356f3a6a23b service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:06.620238 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226feb25-7953-4ed6-8836-e356f3a6a23b service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:06.641883 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:07:06.695694 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6048a7c-7ea0-412c-a57a-a316032b02e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:06.707339 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6048a7c-7ea0-412c-a57a-a316032b02e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:06.707657 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6048a7c-7ea0-412c-a57a-a316032b02e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:06.707984 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6048a7c-7ea0-412c-a57a-a316032b02e6 admin admin] PATH_INFO: `/v3/users/6bb007964d4a4aabbed06ad0afb0aae9` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:06.710499 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6048a7c-7ea0-412c-a57a-a316032b02e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bb007964d4a4aabbed06ad0afb0aae9)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:06.712494 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6048a7c-7ea0-412c-a57a-a316032b02e6 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:06.775976 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8651364-6025-4ae0-b224-cab527c331af admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:06.777405 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e8651364-6025-4ae0-b224-cab527c331af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:06.785521 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e6048a7c-7ea0-412c-a57a-a316032b02e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bb007964d4a4aabbed06ad0afb0aae9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:06.797997 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e6048a7c-7ea0-412c-a57a-a316032b02e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bb007964d4a4aabbed06ad0afb0aae9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:06.799208 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e6048a7c-7ea0-412c-a57a-a316032b02e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bb007964d4a4aabbed06ad0afb0aae9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:06.821124 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e6048a7c-7ea0-412c-a57a-a316032b02e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bb007964d4a4aabbed06ad0afb0aae9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:06.830241 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e6048a7c-7ea0-412c-a57a-a316032b02e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bb007964d4a4aabbed06ad0afb0aae9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:06.839568 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1078/2166] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 04:07:06 2022] DELETE /identity/v3/users/6bb007964d4a4aabbed06ad0afb0aae9 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 04:07:06.858527 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8651364-6025-4ae0-b224-cab527c331af admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:06.910468 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe60546-b927-4349-8d94-c4cc6fe56865 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:06.911106 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe60546-b927-4349-8d94-c4cc6fe56865 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:06.911376 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe60546-b927-4349-8d94-c4cc6fe56865 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:06.911647 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe60546-b927-4349-8d94-c4cc6fe56865 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:06.913449 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-efe60546-b927-4349-8d94-c4cc6fe56865 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:06.979506 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe60546-b927-4349-8d94-c4cc6fe56865 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:06.981057 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe60546-b927-4349-8d94-c4cc6fe56865 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:06.992588 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1089/2167] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:07:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 04:07:07.264273 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6048a7c-7ea0-412c-a57a-a316032b02e6 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:07.265706 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e6048a7c-7ea0-412c-a57a-a316032b02e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:07.283430 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6048a7c-7ea0-412c-a57a-a316032b02e6 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:07.289631 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a628716-5237-457a-af1c-68d08eb46e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:07.290392 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a628716-5237-457a-af1c-68d08eb46e7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:07.290815 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a628716-5237-457a-af1c-68d08eb46e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:07.291167 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a628716-5237-457a-af1c-68d08eb46e7e admin admin] PATH_INFO: `/v3/projects/ae6f3b589faa425fbacb0260387651bf` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:07.292651 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a628716-5237-457a-af1c-68d08eb46e7e admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae6f3b589faa425fbacb0260387651bf)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:07.294752 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a628716-5237-457a-af1c-68d08eb46e7e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:07.375216 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0a628716-5237-457a-af1c-68d08eb46e7e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae6f3b589faa425fbacb0260387651bf', 'request_id': 'req-0a628716-5237-457a-af1c-68d08eb46e7e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:07.382558 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0a628716-5237-457a-af1c-68d08eb46e7e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae6f3b589faa425fbacb0260387651bf', 'request_id': 'req-0a628716-5237-457a-af1c-68d08eb46e7e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:07.393425 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0a628716-5237-457a-af1c-68d08eb46e7e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae6f3b589faa425fbacb0260387651bf', 'request_id': 'req-0a628716-5237-457a-af1c-68d08eb46e7e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:07.408398 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0a628716-5237-457a-af1c-68d08eb46e7e admin admin] The token cache is being invalidate because project ae6f3b589faa425fbacb0260387651bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 04:07:07.409455 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0a628716-5237-457a-af1c-68d08eb46e7e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:07.410451 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0a628716-5237-457a-af1c-68d08eb46e7e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:07.412623 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1079/2168] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 04:07:07 2022] DELETE /identity/v3/projects/ae6f3b589faa425fbacb0260387651bf => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:07:07.416480 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe60546-b927-4349-8d94-c4cc6fe56865 service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:07.429812 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe60546-b927-4349-8d94-c4cc6fe56865 service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:07.451423 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:07:07.511090 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d9086a-7062-45fc-bb26-3a97bd692469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:07.511814 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d9086a-7062-45fc-bb26-3a97bd692469 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:07.512182 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d9086a-7062-45fc-bb26-3a97bd692469 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:07.512593 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d9086a-7062-45fc-bb26-3a97bd692469 admin admin] PATH_INFO: `/v3/users/e50eb73a8048482f9cd2e0185d7b0f29` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:07.513750 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d9086a-7062-45fc-bb26-3a97bd692469 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e50eb73a8048482f9cd2e0185d7b0f29)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:07.516443 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d9086a-7062-45fc-bb26-3a97bd692469 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:07.564323 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a628716-5237-457a-af1c-68d08eb46e7e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:07.564323 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0a628716-5237-457a-af1c-68d08eb46e7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:07.598290 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-86d9086a-7062-45fc-bb26-3a97bd692469 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e50eb73a8048482f9cd2e0185d7b0f29'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:07.600899 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-86d9086a-7062-45fc-bb26-3a97bd692469 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e50eb73a8048482f9cd2e0185d7b0f29'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:07.601138 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-86d9086a-7062-45fc-bb26-3a97bd692469 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e50eb73a8048482f9cd2e0185d7b0f29'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:07.611161 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:07:07.633317 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-86d9086a-7062-45fc-bb26-3a97bd692469 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e50eb73a8048482f9cd2e0185d7b0f29'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:07.642933 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-86d9086a-7062-45fc-bb26-3a97bd692469 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e50eb73a8048482f9cd2e0185d7b0f29'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:07.658838 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1090/2169] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 04:07:07 2022] DELETE /identity/v3/users/e50eb73a8048482f9cd2e0185d7b0f29 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:07:07.662247 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d9086a-7062-45fc-bb26-3a97bd692469 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:07.662247 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-86d9086a-7062-45fc-bb26-3a97bd692469 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:07.688345 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a628716-5237-457a-af1c-68d08eb46e7e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:07.755149 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d9086a-7062-45fc-bb26-3a97bd692469 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:07.769301 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835abbe3-50d1-48c0-90b8-d38b7c09a0a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:07.769967 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835abbe3-50d1-48c0-90b8-d38b7c09a0a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:07.770448 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835abbe3-50d1-48c0-90b8-d38b7c09a0a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:07.770591 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835abbe3-50d1-48c0-90b8-d38b7c09a0a9 admin admin] PATH_INFO: `/v3/users/7f6abf03a1cb4ecaa8b2edfb8a576418` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:07.771758 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835abbe3-50d1-48c0-90b8-d38b7c09a0a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f6abf03a1cb4ecaa8b2edfb8a576418)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:07.773864 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835abbe3-50d1-48c0-90b8-d38b7c09a0a9 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:07.785885 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861999ba-340b-4d6f-b28b-b32d98a939e8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4b9bd741574ddabadd4b7651ebc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:07.787089 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861999ba-340b-4d6f-b28b-b32d98a939e8 service placement] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:07.787447 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861999ba-340b-4d6f-b28b-b32d98a939e8 service placement] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:07.787679 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861999ba-340b-4d6f-b28b-b32d98a939e8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:07.790151 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-861999ba-340b-4d6f-b28b-b32d98a939e8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:07.805874 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861999ba-340b-4d6f-b28b-b32d98a939e8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:07.807800 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861999ba-340b-4d6f-b28b-b32d98a939e8 service placement] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:07.825111 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1080/2170] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:07:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 04:07:07.861547 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:07:07.883146 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-835abbe3-50d1-48c0-90b8-d38b7c09a0a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f6abf03a1cb4ecaa8b2edfb8a576418'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:07.884317 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-835abbe3-50d1-48c0-90b8-d38b7c09a0a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f6abf03a1cb4ecaa8b2edfb8a576418'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:07.885293 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-835abbe3-50d1-48c0-90b8-d38b7c09a0a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f6abf03a1cb4ecaa8b2edfb8a576418'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:07.904571 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-835abbe3-50d1-48c0-90b8-d38b7c09a0a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f6abf03a1cb4ecaa8b2edfb8a576418'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:07.915154 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-835abbe3-50d1-48c0-90b8-d38b7c09a0a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f6abf03a1cb4ecaa8b2edfb8a576418'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:07.935682 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1091/2171] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 04:07:07 2022] DELETE /identity/v3/users/7f6abf03a1cb4ecaa8b2edfb8a576418 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:07:08.018181 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861999ba-340b-4d6f-b28b-b32d98a939e8 service placement] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:08.020124 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-861999ba-340b-4d6f-b28b-b32d98a939e8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:08.049379 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861999ba-340b-4d6f-b28b-b32d98a939e8 service placement] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:08.132732 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af94751b-6347-4ba5-8f8d-8d488039610b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:08.133797 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af94751b-6347-4ba5-8f8d-8d488039610b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:08.134015 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af94751b-6347-4ba5-8f8d-8d488039610b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:08.134383 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af94751b-6347-4ba5-8f8d-8d488039610b admin admin] PATH_INFO: `/v3/projects/b391895835f941a69ebea19b124469b2` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:08.136191 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af94751b-6347-4ba5-8f8d-8d488039610b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b391895835f941a69ebea19b124469b2)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:08.139164 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af94751b-6347-4ba5-8f8d-8d488039610b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:08.156615 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835abbe3-50d1-48c0-90b8-d38b7c09a0a9 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:08.158141 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-835abbe3-50d1-48c0-90b8-d38b7c09a0a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:08.173142 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835abbe3-50d1-48c0-90b8-d38b7c09a0a9 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:08.178249 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4602a1f3-ba24-4b56-be3f-ad1978b6a470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:08.178901 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4602a1f3-ba24-4b56-be3f-ad1978b6a470 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:08.179471 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4602a1f3-ba24-4b56-be3f-ad1978b6a470 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:08.179918 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4602a1f3-ba24-4b56-be3f-ad1978b6a470 admin admin] PATH_INFO: `/v3/projects/909fcdad3cef45ad9cf5494524d2b354` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:08.181087 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4602a1f3-ba24-4b56-be3f-ad1978b6a470 admin admin] RBAC: Authorizing `identity:delete_project(project_id=909fcdad3cef45ad9cf5494524d2b354)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:08.183697 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4602a1f3-ba24-4b56-be3f-ad1978b6a470 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:08.227382 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-af94751b-6347-4ba5-8f8d-8d488039610b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b391895835f941a69ebea19b124469b2', 'request_id': 'req-af94751b-6347-4ba5-8f8d-8d488039610b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:08.235879 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-af94751b-6347-4ba5-8f8d-8d488039610b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b391895835f941a69ebea19b124469b2', 'request_id': 'req-af94751b-6347-4ba5-8f8d-8d488039610b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:08.241241 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:07:08.245921 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-af94751b-6347-4ba5-8f8d-8d488039610b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b391895835f941a69ebea19b124469b2', 'request_id': 'req-af94751b-6347-4ba5-8f8d-8d488039610b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:08.246756 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-4602a1f3-ba24-4b56-be3f-ad1978b6a470 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '909fcdad3cef45ad9cf5494524d2b354', 'request_id': 'req-4602a1f3-ba24-4b56-be3f-ad1978b6a470'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:08.256296 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-4602a1f3-ba24-4b56-be3f-ad1978b6a470 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '909fcdad3cef45ad9cf5494524d2b354', 'request_id': 'req-4602a1f3-ba24-4b56-be3f-ad1978b6a470'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:08.259221 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-af94751b-6347-4ba5-8f8d-8d488039610b admin admin] The token cache is being invalidate because project b391895835f941a69ebea19b124469b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 04:07:08.259541 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-af94751b-6347-4ba5-8f8d-8d488039610b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:08.260386 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-af94751b-6347-4ba5-8f8d-8d488039610b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:08.263042 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1081/2172] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 04:07:08 2022] DELETE /identity/v3/projects/b391895835f941a69ebea19b124469b2 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:07:08.264239 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-4602a1f3-ba24-4b56-be3f-ad1978b6a470 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '909fcdad3cef45ad9cf5494524d2b354', 'request_id': 'req-4602a1f3-ba24-4b56-be3f-ad1978b6a470'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:08.270457 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81df0b-2e59-4ad9-b1aa-4c8dd7cc9ba5 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:08.270844 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81df0b-2e59-4ad9-b1aa-4c8dd7cc9ba5 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:08.271175 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab81df0b-2e59-4ad9-b1aa-4c8dd7cc9ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:08.291689 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-ab81df0b-2e59-4ad9-b1aa-4c8dd7cc9ba5 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:07:08.292431 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-4602a1f3-ba24-4b56-be3f-ad1978b6a470 admin admin] The token cache is being invalidate because project 909fcdad3cef45ad9cf5494524d2b354 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 04:07:08.292760 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-4602a1f3-ba24-4b56-be3f-ad1978b6a470 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:08.292969 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ab81df0b-2e59-4ad9-b1aa-4c8dd7cc9ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:08.293405 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-4602a1f3-ba24-4b56-be3f-ad1978b6a470 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:08.295078 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1092/2173] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 04:07:08 2022] DELETE /identity/v3/projects/909fcdad3cef45ad9cf5494524d2b354 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 04:07:08.301499 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2688cd-0be3-4e9a-9390-c5a4a32aacac None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:08.301791 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2688cd-0be3-4e9a-9390-c5a4a32aacac None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:08.302056 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2688cd-0be3-4e9a-9390-c5a4a32aacac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:08.322961 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-3e2688cd-0be3-4e9a-9390-c5a4a32aacac None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:07:08.324196 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-3e2688cd-0be3-4e9a-9390-c5a4a32aacac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:08.356484 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1082/2174] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:07:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 04:07:08.365534 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8df075-07b2-43d8-84a2-b7151be91c06 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:08.365827 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8df075-07b2-43d8-84a2-b7151be91c06 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:08.366102 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8df075-07b2-43d8-84a2-b7151be91c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:08.385589 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-5a8df075-07b2-43d8-84a2-b7151be91c06 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:07:08.386911 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5a8df075-07b2-43d8-84a2-b7151be91c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:08.387340 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1093/2175] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:07:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 04:07:08.398467 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1083/2176] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:07:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 04:07:08.398662 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7782f23b-8e14-435c-9ec6-1aa6e97841aa None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:08.398662 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7782f23b-8e14-435c-9ec6-1aa6e97841aa None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:08.398662 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7782f23b-8e14-435c-9ec6-1aa6e97841aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:08.399458 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8df075-07b2-43d8-84a2-b7151be91c06 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:08.409265 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8df075-07b2-43d8-84a2-b7151be91c06 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:08.413988 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc29a51-2805-453a-8bd6-cf6c8b5964c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:08.414576 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc29a51-2805-453a-8bd6-cf6c8b5964c6 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:08.414856 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc29a51-2805-453a-8bd6-cf6c8b5964c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:08.415132 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc29a51-2805-453a-8bd6-cf6c8b5964c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:08.416377 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-7782f23b-8e14-435c-9ec6-1aa6e97841aa None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:07:08.416524 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc29a51-2805-453a-8bd6-cf6c8b5964c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 04:07:08.416750 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc29a51-2805-453a-8bd6-cf6c8b5964c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:08.417570 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-7782f23b-8e14-435c-9ec6-1aa6e97841aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:08.418339 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc29a51-2805-453a-8bd6-cf6c8b5964c6 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:08.426169 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1094/2177] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:07:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 04:07:08.430191 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7782f23b-8e14-435c-9ec6-1aa6e97841aa None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:08.433820 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1084/2178] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 04:07:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 04:07:08.434914 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc29a51-2805-453a-8bd6-cf6c8b5964c6 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:08.439249 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7782f23b-8e14-435c-9ec6-1aa6e97841aa None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:08.444059 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabd3ad7-fc59-4547-90ed-17cf0cda70a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:08.444646 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd3ad7-fc59-4547-90ed-17cf0cda70a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:08.444925 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd3ad7-fc59-4547-90ed-17cf0cda70a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:08.445171 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc29a51-2805-453a-8bd6-cf6c8b5964c6 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:08.445306 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd3ad7-fc59-4547-90ed-17cf0cda70a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:08.445818 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabd3ad7-fc59-4547-90ed-17cf0cda70a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 04:07:08.446123 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabd3ad7-fc59-4547-90ed-17cf0cda70a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:08.447697 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabd3ad7-fc59-4547-90ed-17cf0cda70a4 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:08.449977 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e63ff5-2610-42a8-882f-9a83ee81e6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:08.450594 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e63ff5-2610-42a8-882f-9a83ee81e6eb admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:08.450880 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e63ff5-2610-42a8-882f-9a83ee81e6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:08.451185 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e63ff5-2610-42a8-882f-9a83ee81e6eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:08.451819 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e63ff5-2610-42a8-882f-9a83ee81e6eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:08.453799 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e63ff5-2610-42a8-882f-9a83ee81e6eb admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:08.459305 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1095/2179] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 04:07:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 04:07:08.463260 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabd3ad7-fc59-4547-90ed-17cf0cda70a4 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:08.473567 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabd3ad7-fc59-4547-90ed-17cf0cda70a4 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:08.478764 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9daf052-bf38-4289-932b-1a304252f733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:08.479347 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9daf052-bf38-4289-932b-1a304252f733 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:08.479632 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9daf052-bf38-4289-932b-1a304252f733 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:08.479925 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9daf052-bf38-4289-932b-1a304252f733 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:08.480625 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9daf052-bf38-4289-932b-1a304252f733 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:08.482489 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9daf052-bf38-4289-932b-1a304252f733 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:08.482616 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1085/2180] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 04:07:08 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 04:07:08.487288 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e63ff5-2610-42a8-882f-9a83ee81e6eb admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:08.499347 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:07:08.513177 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1096/2181] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 04:07:08 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 04:07:08.517120 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9daf052-bf38-4289-932b-1a304252f733 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:08.552417 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e63ff5-2610-42a8-882f-9a83ee81e6eb admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:08.581294 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9daf052-bf38-4289-932b-1a304252f733 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:08.586634 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3786f53-5f6a-489e-a47f-e22dedfdbeec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:08.587256 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3786f53-5f6a-489e-a47f-e22dedfdbeec admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:08.587541 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3786f53-5f6a-489e-a47f-e22dedfdbeec admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:08.587975 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3786f53-5f6a-489e-a47f-e22dedfdbeec admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:08.588612 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3786f53-5f6a-489e-a47f-e22dedfdbeec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:08.590534 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3786f53-5f6a-489e-a47f-e22dedfdbeec admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:08.607294 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:07:08.618169 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1097/2182] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 04:07:08 2022] POST /identity/v3/users => generated 287 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 04:07:08.621745 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4583d69-91b6-4f2d-bbed-c07214603ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:08.621936 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3786f53-5f6a-489e-a47f-e22dedfdbeec admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:08.622501 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4583d69-91b6-4f2d-bbed-c07214603ca9 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:08.622788 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4583d69-91b6-4f2d-bbed-c07214603ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:08.623063 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4583d69-91b6-4f2d-bbed-c07214603ca9 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:08.623974 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4583d69-91b6-4f2d-bbed-c07214603ca9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:08.627700 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4583d69-91b6-4f2d-bbed-c07214603ca9 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:08.632942 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3786f53-5f6a-489e-a47f-e22dedfdbeec admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:08.638019 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a17e6-2ec2-4cb4-ba8a-35c33159e852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:08.638574 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a17e6-2ec2-4cb4-ba8a-35c33159e852 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:08.638857 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a17e6-2ec2-4cb4-ba8a-35c33159e852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:08.639193 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54a17e6-2ec2-4cb4-ba8a-35c33159e852 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:08.639844 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54a17e6-2ec2-4cb4-ba8a-35c33159e852 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:08.642339 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54a17e6-2ec2-4cb4-ba8a-35c33159e852 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:08.658824 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1098/2183] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:07:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 04:07:08.660408 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a17e6-2ec2-4cb4-ba8a-35c33159e852 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:08.661510 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1086/2184] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 04:07:08 2022] POST /identity/v3/users => generated 286 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 04:07:08.666859 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4583d69-91b6-4f2d-bbed-c07214603ca9 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:08.671009 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54a17e6-2ec2-4cb4-ba8a-35c33159e852 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:08.677365 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e095abc-4721-4b89-b5ab-ffa269536ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:08.678100 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e095abc-4721-4b89-b5ab-ffa269536ae3 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:08.678454 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e095abc-4721-4b89-b5ab-ffa269536ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:08.679056 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e095abc-4721-4b89-b5ab-ffa269536ae3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:08.679517 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e095abc-4721-4b89-b5ab-ffa269536ae3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:08.679675 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4583d69-91b6-4f2d-bbed-c07214603ca9 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:08.684860 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3caaad-ae37-483e-9438-3796f0a5b3bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:08.688374 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e095abc-4721-4b89-b5ab-ffa269536ae3 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:08.689090 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3caaad-ae37-483e-9438-3796f0a5b3bf admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:08.689504 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3caaad-ae37-483e-9438-3796f0a5b3bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:08.689801 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3caaad-ae37-483e-9438-3796f0a5b3bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:08.690538 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3caaad-ae37-483e-9438-3796f0a5b3bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:08.692224 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3caaad-ae37-483e-9438-3796f0a5b3bf admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:08.704538 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1087/2185] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:07:08 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:07:08.707583 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1099/2186] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:07:08 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:07:08.708841 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e095abc-4721-4b89-b5ab-ffa269536ae3 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:08.708993 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3caaad-ae37-483e-9438-3796f0a5b3bf admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:08.720744 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e095abc-4721-4b89-b5ab-ffa269536ae3 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:08.723322 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3caaad-ae37-483e-9438-3796f0a5b3bf admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:08.725988 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93ce865-5a2c-498b-b7e9-7fce0e3afc06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:08.726847 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ce865-5a2c-498b-b7e9-7fce0e3afc06 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:08.727119 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ce865-5a2c-498b-b7e9-7fce0e3afc06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:08.727465 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ce865-5a2c-498b-b7e9-7fce0e3afc06 admin admin] PATH_INFO: `/v3/projects/3e61d1b98c0b474a95a99962af8b20b5/users/7f89c4e2fa8c495fa8bc7f64a3635fe0/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:08.728335 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987a75b3-7817-42fa-b6c5-ee448fcf1428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:08.728884 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987a75b3-7817-42fa-b6c5-ee448fcf1428 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:08.729204 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987a75b3-7817-42fa-b6c5-ee448fcf1428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:08.729481 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987a75b3-7817-42fa-b6c5-ee448fcf1428 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:08.730134 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987a75b3-7817-42fa-b6c5-ee448fcf1428 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:08.731725 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987a75b3-7817-42fa-b6c5-ee448fcf1428 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:08.743724 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1088/2187] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:07:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:07:08.744620 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93ce865-5a2c-498b-b7e9-7fce0e3afc06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e61d1b98c0b474a95a99962af8b20b5, user_id=7f89c4e2fa8c495fa8bc7f64a3635fe0, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:08.747426 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93ce865-5a2c-498b-b7e9-7fce0e3afc06 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:08.748028 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987a75b3-7817-42fa-b6c5-ee448fcf1428 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:08.759758 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987a75b3-7817-42fa-b6c5-ee448fcf1428 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:08.763521 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1100/2188] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 04:07:08 2022] PUT /identity/v3/projects/3e61d1b98c0b474a95a99962af8b20b5/users/7f89c4e2fa8c495fa8bc7f64a3635fe0/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:07:08.768871 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0fd1b3-180d-4322-a579-d0d22f7e8dc5 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:08.769162 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0fd1b3-180d-4322-a579-d0d22f7e8dc5 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:08.769432 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0fd1b3-180d-4322-a579-d0d22f7e8dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:08.792255 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-3f0fd1b3-180d-4322-a579-d0d22f7e8dc5 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:07:08.793951 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-3f0fd1b3-180d-4322-a579-d0d22f7e8dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:08.828633 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e381d-ee8a-4210-9814-7973779cfc5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:08.828633 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e381d-ee8a-4210-9814-7973779cfc5a admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:08.828633 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e381d-ee8a-4210-9814-7973779cfc5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:08.828633 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80e381d-ee8a-4210-9814-7973779cfc5a admin admin] PATH_INFO: `/v3/projects/0004cf68384d4475a1c41f13103afe52/users/dcd20d5ba9ca47f8b8389f79d0215bc7/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:08.845325 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80e381d-ee8a-4210-9814-7973779cfc5a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0004cf68384d4475a1c41f13103afe52, user_id=dcd20d5ba9ca47f8b8389f79d0215bc7, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:08.849582 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80e381d-ee8a-4210-9814-7973779cfc5a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:08.864538 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1089/2189] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 04:07:08 2022] PUT /identity/v3/projects/0004cf68384d4475a1c41f13103afe52/users/dcd20d5ba9ca47f8b8389f79d0215bc7/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:07:08.874776 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1101/2190] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:07:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 04:07:08.875896 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e381d-ee8a-4210-9814-7973779cfc5a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:08.877640 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e80e381d-ee8a-4210-9814-7973779cfc5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:08.880185 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411837be-0d36-4133-9da9-011474bca4e6 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:08.880479 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411837be-0d36-4133-9da9-011474bca4e6 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:08.880759 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411837be-0d36-4133-9da9-011474bca4e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:08.910029 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-411837be-0d36-4133-9da9-011474bca4e6 None None] MFA Rules not processed for user `dcd20d5ba9ca47f8b8389f79d0215bc7`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:07:08.911164 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-411837be-0d36-4133-9da9-011474bca4e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:08.935809 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80e381d-ee8a-4210-9814-7973779cfc5a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:08.939875 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:07:08.940141 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f669339c-e491-4e39-845a-5caf32456fe3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:08.940903 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f669339c-e491-4e39-845a-5caf32456fe3 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:08.941080 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f669339c-e491-4e39-845a-5caf32456fe3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:08.941308 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f669339c-e491-4e39-845a-5caf32456fe3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:08.977728 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1102/2191] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:07:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:07:08.989042 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411837be-0d36-4133-9da9-011474bca4e6 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:08.990473 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-411837be-0d36-4133-9da9-011474bca4e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:09.003407 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:07:09.008393 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f669339c-e491-4e39-845a-5caf32456fe3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:09.010157 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f669339c-e491-4e39-845a-5caf32456fe3 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:09.020612 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1090/2192] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:07:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 04:07:09.051525 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411837be-0d36-4133-9da9-011474bca4e6 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:09.055085 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d03087f-c94b-4d9c-901e-6e252c52f8e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:09.055699 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d03087f-c94b-4d9c-901e-6e252c52f8e4 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:09.056021 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d03087f-c94b-4d9c-901e-6e252c52f8e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:09.056306 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d03087f-c94b-4d9c-901e-6e252c52f8e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:09.066536 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d03087f-c94b-4d9c-901e-6e252c52f8e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:09.068088 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d03087f-c94b-4d9c-901e-6e252c52f8e4 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:09.077511 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1103/2193] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:07:08 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 04:07:09.100168 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f669339c-e491-4e39-845a-5caf32456fe3 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:09.109953 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f669339c-e491-4e39-845a-5caf32456fe3 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:09.115021 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44638afd-82a9-4aa1-8ead-e9d6e28033f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:09.115643 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44638afd-82a9-4aa1-8ead-e9d6e28033f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:09.115923 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44638afd-82a9-4aa1-8ead-e9d6e28033f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:09.116234 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44638afd-82a9-4aa1-8ead-e9d6e28033f8 admin admin] PATH_INFO: `/v3/users/dcd20d5ba9ca47f8b8389f79d0215bc7` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:09.117378 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44638afd-82a9-4aa1-8ead-e9d6e28033f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcd20d5ba9ca47f8b8389f79d0215bc7)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:09.119305 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44638afd-82a9-4aa1-8ead-e9d6e28033f8 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:09.173819 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-44638afd-82a9-4aa1-8ead-e9d6e28033f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcd20d5ba9ca47f8b8389f79d0215bc7'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:09.181404 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-44638afd-82a9-4aa1-8ead-e9d6e28033f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcd20d5ba9ca47f8b8389f79d0215bc7'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:09.182115 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-44638afd-82a9-4aa1-8ead-e9d6e28033f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcd20d5ba9ca47f8b8389f79d0215bc7'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:09.195265 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-44638afd-82a9-4aa1-8ead-e9d6e28033f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcd20d5ba9ca47f8b8389f79d0215bc7'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:09.207997 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-44638afd-82a9-4aa1-8ead-e9d6e28033f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcd20d5ba9ca47f8b8389f79d0215bc7'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:09.215546 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1091/2194] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 04:07:09 2022] DELETE /identity/v3/users/dcd20d5ba9ca47f8b8389f79d0215bc7 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:07:09.219968 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade424f0-7329-445a-85fb-ece89a333edf None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:09.220469 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade424f0-7329-445a-85fb-ece89a333edf None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:09.220744 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade424f0-7329-445a-85fb-ece89a333edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:09.241170 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-ade424f0-7329-445a-85fb-ece89a333edf None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:07:09.242533 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ade424f0-7329-445a-85fb-ece89a333edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:09.307505 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1104/2195] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:07:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:07:09.320528 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44638afd-82a9-4aa1-8ead-e9d6e28033f8 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:09.322458 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-44638afd-82a9-4aa1-8ead-e9d6e28033f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:09.413052 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44638afd-82a9-4aa1-8ead-e9d6e28033f8 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:09.417152 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f5620d-571c-4775-9456-5eec9d5dfbbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:09.417828 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f5620d-571c-4775-9456-5eec9d5dfbbb service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:09.418114 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f5620d-571c-4775-9456-5eec9d5dfbbb service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:09.418388 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f5620d-571c-4775-9456-5eec9d5dfbbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:09.430286 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f5620d-571c-4775-9456-5eec9d5dfbbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:09.431558 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f5620d-571c-4775-9456-5eec9d5dfbbb service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:09.446501 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1092/2196] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:07:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 04:07:09.805057 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade424f0-7329-445a-85fb-ece89a333edf None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:09.806506 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ade424f0-7329-445a-85fb-ece89a333edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:09.828724 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade424f0-7329-445a-85fb-ece89a333edf None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:09.843020 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa46c67-2b57-4ecc-ae4b-2cd60d031bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:09.843866 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa46c67-2b57-4ecc-ae4b-2cd60d031bf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:09.844181 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa46c67-2b57-4ecc-ae4b-2cd60d031bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:09.844959 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa46c67-2b57-4ecc-ae4b-2cd60d031bf7 admin admin] PATH_INFO: `/v3/projects/0004cf68384d4475a1c41f13103afe52` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:09.846054 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa46c67-2b57-4ecc-ae4b-2cd60d031bf7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0004cf68384d4475a1c41f13103afe52)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:09.848860 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa46c67-2b57-4ecc-ae4b-2cd60d031bf7 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:09.920367 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-baa46c67-2b57-4ecc-ae4b-2cd60d031bf7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0004cf68384d4475a1c41f13103afe52', 'request_id': 'req-baa46c67-2b57-4ecc-ae4b-2cd60d031bf7'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:09.929679 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-baa46c67-2b57-4ecc-ae4b-2cd60d031bf7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0004cf68384d4475a1c41f13103afe52', 'request_id': 'req-baa46c67-2b57-4ecc-ae4b-2cd60d031bf7'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:09.937961 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-baa46c67-2b57-4ecc-ae4b-2cd60d031bf7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0004cf68384d4475a1c41f13103afe52', 'request_id': 'req-baa46c67-2b57-4ecc-ae4b-2cd60d031bf7'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:09.942878 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:07:09.960114 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-baa46c67-2b57-4ecc-ae4b-2cd60d031bf7 admin admin] The token cache is being invalidate because project 0004cf68384d4475a1c41f13103afe52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 04:07:09.960488 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-baa46c67-2b57-4ecc-ae4b-2cd60d031bf7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:09.961300 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-baa46c67-2b57-4ecc-ae4b-2cd60d031bf7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:09.963337 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1105/2197] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 04:07:09 2022] DELETE /identity/v3/projects/0004cf68384d4475a1c41f13103afe52 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:07:09.968165 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4216c758-0d15-40b2-8774-382583acc24c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:09.968720 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4216c758-0d15-40b2-8774-382583acc24c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:09.968720 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4216c758-0d15-40b2-8774-382583acc24c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:09.997129 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-4216c758-0d15-40b2-8774-382583acc24c None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:07:09.998890 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4216c758-0d15-40b2-8774-382583acc24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:10.014050 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa46c67-2b57-4ecc-ae4b-2cd60d031bf7 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:10.016578 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-baa46c67-2b57-4ecc-ae4b-2cd60d031bf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:10.092195 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1093/2198] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:07:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:07:10.100768 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70802a73-f1c0-4d09-941d-50cd965ae2dd None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:10.101055 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70802a73-f1c0-4d09-941d-50cd965ae2dd None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:10.101404 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70802a73-f1c0-4d09-941d-50cd965ae2dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:10.107366 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa46c67-2b57-4ecc-ae4b-2cd60d031bf7 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:10.115567 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff60c2f-f30c-482c-96ed-c94a430bdd5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:10.116564 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff60c2f-f30c-482c-96ed-c94a430bdd5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:10.117260 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff60c2f-f30c-482c-96ed-c94a430bdd5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:10.117543 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff60c2f-f30c-482c-96ed-c94a430bdd5b admin admin] PATH_INFO: `/v3/users/6b04263cfd364bb19da81fa9b1a10c11` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:10.118743 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff60c2f-f30c-482c-96ed-c94a430bdd5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b04263cfd364bb19da81fa9b1a10c11)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:10.122041 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff60c2f-f30c-482c-96ed-c94a430bdd5b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:10.147447 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-70802a73-f1c0-4d09-941d-50cd965ae2dd None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:07:10.148733 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-70802a73-f1c0-4d09-941d-50cd965ae2dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:10.162284 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1094/2199] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:07:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:07:10.167147 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70802a73-f1c0-4d09-941d-50cd965ae2dd None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:10.173713 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:07:10.192260 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70802a73-f1c0-4d09-941d-50cd965ae2dd None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:10.197203 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c567f1ba-de73-4840-92b2-b7e91c967d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:10.197788 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567f1ba-de73-4840-92b2-b7e91c967d1d admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:10.198392 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567f1ba-de73-4840-92b2-b7e91c967d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:10.198392 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567f1ba-de73-4840-92b2-b7e91c967d1d admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:10.199128 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c567f1ba-de73-4840-92b2-b7e91c967d1d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 04:07:10.199555 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c567f1ba-de73-4840-92b2-b7e91c967d1d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:10.201212 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c567f1ba-de73-4840-92b2-b7e91c967d1d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:10.218160 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1095/2200] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 04:07:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 04:07:10.219885 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c567f1ba-de73-4840-92b2-b7e91c967d1d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:10.221308 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1ff60c2f-f30c-482c-96ed-c94a430bdd5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b04263cfd364bb19da81fa9b1a10c11'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:10.226657 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1ff60c2f-f30c-482c-96ed-c94a430bdd5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b04263cfd364bb19da81fa9b1a10c11'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:10.227542 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1ff60c2f-f30c-482c-96ed-c94a430bdd5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b04263cfd364bb19da81fa9b1a10c11'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:10.230810 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c567f1ba-de73-4840-92b2-b7e91c967d1d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:10.236898 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dafc66-4520-4718-9cfd-54d7fda66741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:10.237541 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dafc66-4520-4718-9cfd-54d7fda66741 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:10.237825 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dafc66-4520-4718-9cfd-54d7fda66741 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:10.238097 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dafc66-4520-4718-9cfd-54d7fda66741 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:10.238991 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dafc66-4520-4718-9cfd-54d7fda66741 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:10.241411 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dafc66-4520-4718-9cfd-54d7fda66741 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:10.257850 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1ff60c2f-f30c-482c-96ed-c94a430bdd5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b04263cfd364bb19da81fa9b1a10c11'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:10.272898 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1ff60c2f-f30c-482c-96ed-c94a430bdd5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b04263cfd364bb19da81fa9b1a10c11'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:10.291167 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1096/2201] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 04:07:10 2022] POST /identity/v3/projects => generated 383 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:07:10.295765 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dafc66-4520-4718-9cfd-54d7fda66741 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:10.299024 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b5dafc66-4520-4718-9cfd-54d7fda66741 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:10.303074 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1106/2202] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 04:07:10 2022] DELETE /identity/v3/users/6b04263cfd364bb19da81fa9b1a10c11 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 04:07:10.304643 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd75f6-f4a9-404a-8a14-323035e096b8 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:10.304924 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd75f6-f4a9-404a-8a14-323035e096b8 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:10.305374 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3bd75f6-f4a9-404a-8a14-323035e096b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:10.351219 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-e3bd75f6-f4a9-404a-8a14-323035e096b8 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:07:10.352502 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e3bd75f6-f4a9-404a-8a14-323035e096b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:10.432620 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dafc66-4520-4718-9cfd-54d7fda66741 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:10.433931 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1107/2203] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:07:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:07:10.443765 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91b0450-f2e2-467a-97d2-90e79ceaf313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:10.444512 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b0450-f2e2-467a-97d2-90e79ceaf313 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:10.444915 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b0450-f2e2-467a-97d2-90e79ceaf313 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:10.445259 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91b0450-f2e2-467a-97d2-90e79ceaf313 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:10.445983 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91b0450-f2e2-467a-97d2-90e79ceaf313 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:10.448579 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91b0450-f2e2-467a-97d2-90e79ceaf313 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:10.450118 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bd75f6-f4a9-404a-8a14-323035e096b8 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:10.451906 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e3bd75f6-f4a9-404a-8a14-323035e096b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:10.485999 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1097/2204] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 04:07:10 2022] POST /identity/v3/users => generated 292 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:07:10.487832 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91b0450-f2e2-467a-97d2-90e79ceaf313 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:10.491534 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:07:10.511240 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91b0450-f2e2-467a-97d2-90e79ceaf313 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:10.523231 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa1b630-4df9-4e42-a34b-9abad446d657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:10.524051 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1b630-4df9-4e42-a34b-9abad446d657 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:10.524406 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1b630-4df9-4e42-a34b-9abad446d657 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:10.525069 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa1b630-4df9-4e42-a34b-9abad446d657 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:10.527091 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa1b630-4df9-4e42-a34b-9abad446d657 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:10.532326 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa1b630-4df9-4e42-a34b-9abad446d657 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:10.557753 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1098/2205] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:07:10 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:07:10.559741 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa1b630-4df9-4e42-a34b-9abad446d657 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:10.569623 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3bd75f6-f4a9-404a-8a14-323035e096b8 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:10.575674 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4505a995-27d4-4762-bfe4-c2be724f0ee2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:10.576492 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4505a995-27d4-4762-bfe4-c2be724f0ee2 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:10.576841 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4505a995-27d4-4762-bfe4-c2be724f0ee2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:10.577537 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4505a995-27d4-4762-bfe4-c2be724f0ee2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:10.585243 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa1b630-4df9-4e42-a34b-9abad446d657 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:10.592861 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4505a995-27d4-4762-bfe4-c2be724f0ee2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:10.595260 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4505a995-27d4-4762-bfe4-c2be724f0ee2 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:10.599410 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a7f3ea-a51f-46c0-a96b-f30918c8431d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:10.600446 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a7f3ea-a51f-46c0-a96b-f30918c8431d admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:10.600737 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a7f3ea-a51f-46c0-a96b-f30918c8431d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:10.601012 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a7f3ea-a51f-46c0-a96b-f30918c8431d admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:10.602274 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a7f3ea-a51f-46c0-a96b-f30918c8431d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:10.605655 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a7f3ea-a51f-46c0-a96b-f30918c8431d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:10.612043 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1108/2206] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:07:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 04:07:10.635318 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4505a995-27d4-4762-bfe4-c2be724f0ee2 service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:10.636220 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1099/2207] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:07:10 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 04:07:10.652384 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4505a995-27d4-4762-bfe4-c2be724f0ee2 service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:10.659754 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94e2fd7-86b4-4225-8c16-9cb4561604be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:10.660800 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94e2fd7-86b4-4225-8c16-9cb4561604be admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:10.661162 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94e2fd7-86b4-4225-8c16-9cb4561604be admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:10.661557 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94e2fd7-86b4-4225-8c16-9cb4561604be admin admin] PATH_INFO: `/v3/projects/4261bef348d94cfc9ef05ccc990984b2/users/ddec8a4305d6441a8b19f184543d997f/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:10.684860 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94e2fd7-86b4-4225-8c16-9cb4561604be admin admin] RBAC: Authorizing `identity:create_grant(project_id=4261bef348d94cfc9ef05ccc990984b2, user_id=ddec8a4305d6441a8b19f184543d997f, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:10.690452 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94e2fd7-86b4-4225-8c16-9cb4561604be admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:10.711313 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1109/2208] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 04:07:10 2022] PUT /identity/v3/projects/4261bef348d94cfc9ef05ccc990984b2/users/ddec8a4305d6441a8b19f184543d997f/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 04:07:10.722380 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acebd6e7-0842-4580-99c2-a83b04ab98df None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:10.722670 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acebd6e7-0842-4580-99c2-a83b04ab98df None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:10.723000 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acebd6e7-0842-4580-99c2-a83b04ab98df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:10.764246 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-acebd6e7-0842-4580-99c2-a83b04ab98df None None] MFA Rules not processed for user `ddec8a4305d6441a8b19f184543d997f`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:07:10.765967 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-acebd6e7-0842-4580-99c2-a83b04ab98df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:10.904170 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1100/2209] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:07:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:07:10.908855 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94e2fd7-86b4-4225-8c16-9cb4561604be admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:11.007794 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94e2fd7-86b4-4225-8c16-9cb4561604be admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:11.012821 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4130568-9767-464b-bbd7-93666957ea8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:11.013460 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4130568-9767-464b-bbd7-93666957ea8c admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:11.013746 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4130568-9767-464b-bbd7-93666957ea8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:11.014022 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4130568-9767-464b-bbd7-93666957ea8c admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:11.014736 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4130568-9767-464b-bbd7-93666957ea8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:11.017552 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4130568-9767-464b-bbd7-93666957ea8c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:11.055346 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1110/2210] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 04:07:10 2022] POST /identity/v3/projects => generated 383 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 04:07:11.059392 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acebd6e7-0842-4580-99c2-a83b04ab98df None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:11.089441 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acebd6e7-0842-4580-99c2-a83b04ab98df None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:11.113408 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:07:11.195532 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7861ea6c-8b56-4526-8afe-4496564879bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:11.198144 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7861ea6c-8b56-4526-8afe-4496564879bb admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:11.200260 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7861ea6c-8b56-4526-8afe-4496564879bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:11.201628 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7861ea6c-8b56-4526-8afe-4496564879bb admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:11.202454 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7861ea6c-8b56-4526-8afe-4496564879bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:11.206606 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7861ea6c-8b56-4526-8afe-4496564879bb admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:11.253336 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4130568-9767-464b-bbd7-93666957ea8c admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:11.256084 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1101/2211] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 04:07:11 2022] POST /identity/v3/users => generated 292 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:07:11.267588 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4130568-9767-464b-bbd7-93666957ea8c admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:11.274105 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2574b0f5-5abf-413d-b46a-413c88944745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:11.274857 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2574b0f5-5abf-413d-b46a-413c88944745 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:11.275207 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2574b0f5-5abf-413d-b46a-413c88944745 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:11.275551 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2574b0f5-5abf-413d-b46a-413c88944745 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:11.276402 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2574b0f5-5abf-413d-b46a-413c88944745 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:11.278300 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2574b0f5-5abf-413d-b46a-413c88944745 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:11.304054 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1111/2212] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:07:11 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:07:11.306215 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7861ea6c-8b56-4526-8afe-4496564879bb admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:11.319678 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7861ea6c-8b56-4526-8afe-4496564879bb admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:11.335406 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e33625-ae06-4ea5-8029-9ef5a04bf924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:11.336091 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e33625-ae06-4ea5-8029-9ef5a04bf924 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:11.336489 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e33625-ae06-4ea5-8029-9ef5a04bf924 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:11.336891 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e33625-ae06-4ea5-8029-9ef5a04bf924 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:11.337477 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e33625-ae06-4ea5-8029-9ef5a04bf924 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:11.339138 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e33625-ae06-4ea5-8029-9ef5a04bf924 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:11.360145 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1102/2213] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:07:11 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:07:11.364507 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2574b0f5-5abf-413d-b46a-413c88944745 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:11.379752 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2574b0f5-5abf-413d-b46a-413c88944745 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:11.387330 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd505b9e-d460-4fa9-a787-1b1213f1084b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:11.387984 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd505b9e-d460-4fa9-a787-1b1213f1084b admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:11.388488 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd505b9e-d460-4fa9-a787-1b1213f1084b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:11.388830 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd505b9e-d460-4fa9-a787-1b1213f1084b admin admin] PATH_INFO: `/v3/projects/551ee82723344f0e9f08f355de828bc2/users/639cc6fb33f045cdb32591db0b89751c/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:11.413612 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd505b9e-d460-4fa9-a787-1b1213f1084b admin admin] RBAC: Authorizing `identity:create_grant(project_id=551ee82723344f0e9f08f355de828bc2, user_id=639cc6fb33f045cdb32591db0b89751c, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:11.425031 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd505b9e-d460-4fa9-a787-1b1213f1084b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:11.455848 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1112/2214] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 04:07:11 2022] PUT /identity/v3/projects/551ee82723344f0e9f08f355de828bc2/users/639cc6fb33f045cdb32591db0b89751c/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:07:11.466606 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e425d83-f28b-4e37-b865-7d991c0a9e1f None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:11.466824 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e425d83-f28b-4e37-b865-7d991c0a9e1f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:11.467069 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e425d83-f28b-4e37-b865-7d991c0a9e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:11.511283 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-4e425d83-f28b-4e37-b865-7d991c0a9e1f None None] MFA Rules not processed for user `639cc6fb33f045cdb32591db0b89751c`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:07:11.512771 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4e425d83-f28b-4e37-b865-7d991c0a9e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:11.625936 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1103/2215] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:07:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:07:11.628214 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd505b9e-d460-4fa9-a787-1b1213f1084b admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:11.630225 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e425d83-f28b-4e37-b865-7d991c0a9e1f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:11.630979 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-bd505b9e-d460-4fa9-a787-1b1213f1084b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:11.738592 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e425d83-f28b-4e37-b865-7d991c0a9e1f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:11.743903 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:07:11.744496 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7611967-ad36-489b-8079-3b6717863efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:11.745368 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7611967-ad36-489b-8079-3b6717863efe admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:11.745726 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7611967-ad36-489b-8079-3b6717863efe admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:11.746078 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7611967-ad36-489b-8079-3b6717863efe admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:11.746933 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7611967-ad36-489b-8079-3b6717863efe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:11.749496 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7611967-ad36-489b-8079-3b6717863efe admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:11.789291 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd505b9e-d460-4fa9-a787-1b1213f1084b admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:11.801720 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1104/2216] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 04:07:11 2022] POST /identity/v3/projects => generated 383 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:07:11.805384 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e56615-16d8-4d92-acf4-140969e9a6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:11.806174 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7611967-ad36-489b-8079-3b6717863efe admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:11.806314 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e56615-16d8-4d92-acf4-140969e9a6c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:11.806456 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e56615-16d8-4d92-acf4-140969e9a6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:11.807029 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e56615-16d8-4d92-acf4-140969e9a6c0 admin admin] PATH_INFO: `/v3/projects/38054306f6bf4fcbb5215710799a5fc3` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:11.808648 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e56615-16d8-4d92-acf4-140969e9a6c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38054306f6bf4fcbb5215710799a5fc3)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:11.810585 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e56615-16d8-4d92-acf4-140969e9a6c0 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:11.834215 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:07:11.934722 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-c4e56615-16d8-4d92-acf4-140969e9a6c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38054306f6bf4fcbb5215710799a5fc3', 'request_id': 'req-c4e56615-16d8-4d92-acf4-140969e9a6c0'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:11.939115 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7611967-ad36-489b-8079-3b6717863efe admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:11.947330 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-c4e56615-16d8-4d92-acf4-140969e9a6c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38054306f6bf4fcbb5215710799a5fc3', 'request_id': 'req-c4e56615-16d8-4d92-acf4-140969e9a6c0'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:11.964464 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-c4e56615-16d8-4d92-acf4-140969e9a6c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38054306f6bf4fcbb5215710799a5fc3', 'request_id': 'req-c4e56615-16d8-4d92-acf4-140969e9a6c0'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:12.007805 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-c4e56615-16d8-4d92-acf4-140969e9a6c0 admin admin] The token cache is being invalidate because project 38054306f6bf4fcbb5215710799a5fc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 04:07:12.008529 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-c4e56615-16d8-4d92-acf4-140969e9a6c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:12.011004 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-c4e56615-16d8-4d92-acf4-140969e9a6c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:12.013349 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1113/2217] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 04:07:11 2022] DELETE /identity/v3/projects/38054306f6bf4fcbb5215710799a5fc3 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:07:12.022535 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e629efb2-d938-48ef-943d-2cc1adf5e137 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:12.023234 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e629efb2-d938-48ef-943d-2cc1adf5e137 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:12.023739 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e629efb2-d938-48ef-943d-2cc1adf5e137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:12.089705 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05935699-4818-4ff0-b505-d523175017ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:12.090355 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05935699-4818-4ff0-b505-d523175017ef admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:12.090632 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05935699-4818-4ff0-b505-d523175017ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:12.090903 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05935699-4818-4ff0-b505-d523175017ef admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:12.091752 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05935699-4818-4ff0-b505-d523175017ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:12.094021 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05935699-4818-4ff0-b505-d523175017ef admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:12.095698 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-e629efb2-d938-48ef-943d-2cc1adf5e137 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:07:12.097143 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e629efb2-d938-48ef-943d-2cc1adf5e137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:12.128000 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1114/2218] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:07:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:07:12.128243 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1105/2219] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 04:07:11 2022] POST /identity/v3/users => generated 298 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 04:07:12.131210 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e629efb2-d938-48ef-943d-2cc1adf5e137 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:12.133518 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c663d5d9-5b41-4b21-a17c-2c517c8ca752 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:12.134179 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c663d5d9-5b41-4b21-a17c-2c517c8ca752 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:12.134518 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c663d5d9-5b41-4b21-a17c-2c517c8ca752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:12.134702 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e629efb2-d938-48ef-943d-2cc1adf5e137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:12.159100 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e629efb2-d938-48ef-943d-2cc1adf5e137 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:12.163322 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-c663d5d9-5b41-4b21-a17c-2c517c8ca752 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:07:12.164856 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c663d5d9-5b41-4b21-a17c-2c517c8ca752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:12.165350 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a16f01-a011-47ef-a8c2-b8eead838c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:12.166085 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a16f01-a011-47ef-a8c2-b8eead838c53 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:12.166518 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a16f01-a011-47ef-a8c2-b8eead838c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:12.166888 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a16f01-a011-47ef-a8c2-b8eead838c53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:12.167760 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a16f01-a011-47ef-a8c2-b8eead838c53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:12.169716 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a16f01-a011-47ef-a8c2-b8eead838c53 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:12.180107 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1106/2220] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:07:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:07:12.181461 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c663d5d9-5b41-4b21-a17c-2c517c8ca752 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:12.190151 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1115/2221] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:07:12 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:07:12.194474 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a16f01-a011-47ef-a8c2-b8eead838c53 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:12.201368 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c663d5d9-5b41-4b21-a17c-2c517c8ca752 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:12.209456 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ecbc9b-82ef-45ed-90c0-baea5aa14e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:12.210142 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ecbc9b-82ef-45ed-90c0-baea5aa14e29 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:12.210422 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ecbc9b-82ef-45ed-90c0-baea5aa14e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:12.211006 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ecbc9b-82ef-45ed-90c0-baea5aa14e29 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:12.211669 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ecbc9b-82ef-45ed-90c0-baea5aa14e29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 04:07:12.212036 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ecbc9b-82ef-45ed-90c0-baea5aa14e29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:12.213442 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a16f01-a011-47ef-a8c2-b8eead838c53 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:12.214634 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ecbc9b-82ef-45ed-90c0-baea5aa14e29 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:12.219673 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ac3dfc-31e0-48af-8f05-20d0c33accfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:12.220504 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac3dfc-31e0-48af-8f05-20d0c33accfc admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:12.220880 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac3dfc-31e0-48af-8f05-20d0c33accfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:12.221261 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac3dfc-31e0-48af-8f05-20d0c33accfc admin admin] PATH_INFO: `/v3/projects/da458d79154c42a7821ead0b5d2e30ae/users/7dfc57d3f7fe4089aacb85ecfdcfaf2b/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:12.235189 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1107/2222] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 04:07:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 04:07:12.236251 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ecbc9b-82ef-45ed-90c0-baea5aa14e29 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:12.257564 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ecbc9b-82ef-45ed-90c0-baea5aa14e29 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:12.258303 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ac3dfc-31e0-48af-8f05-20d0c33accfc admin admin] RBAC: Authorizing `identity:create_grant(project_id=da458d79154c42a7821ead0b5d2e30ae, user_id=7dfc57d3f7fe4089aacb85ecfdcfaf2b, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:12.262507 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ac3dfc-31e0-48af-8f05-20d0c33accfc admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:12.267354 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8eb6c0-b045-4903-86c8-6c04c15dadf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:12.268263 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8eb6c0-b045-4903-86c8-6c04c15dadf1 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:12.268726 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8eb6c0-b045-4903-86c8-6c04c15dadf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:12.269168 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8eb6c0-b045-4903-86c8-6c04c15dadf1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:12.270701 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8eb6c0-b045-4903-86c8-6c04c15dadf1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:12.272991 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8eb6c0-b045-4903-86c8-6c04c15dadf1 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:12.287294 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1116/2223] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 04:07:12 2022] PUT /identity/v3/projects/da458d79154c42a7821ead0b5d2e30ae/users/7dfc57d3f7fe4089aacb85ecfdcfaf2b/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:07:12.288422 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ac3dfc-31e0-48af-8f05-20d0c33accfc admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:12.321660 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1108/2224] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 04:07:12 2022] POST /identity/v3/projects => generated 371 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 04:07:12.325899 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8eb6c0-b045-4903-86c8-6c04c15dadf1 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:12.400010 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ac3dfc-31e0-48af-8f05-20d0c33accfc admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:12.457368 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8eb6c0-b045-4903-86c8-6c04c15dadf1 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:12.466123 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924a5393-d0b0-4aec-9ec0-00d69ce4a800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:12.466752 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a5393-d0b0-4aec-9ec0-00d69ce4a800 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:12.467036 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a5393-d0b0-4aec-9ec0-00d69ce4a800 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:12.467869 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924a5393-d0b0-4aec-9ec0-00d69ce4a800 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:12.469178 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924a5393-d0b0-4aec-9ec0-00d69ce4a800 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:12.470507 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:07:12.471733 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924a5393-d0b0-4aec-9ec0-00d69ce4a800 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:12.514451 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1109/2225] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 04:07:12 2022] POST /identity/v3/users => generated 286 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:07:12.514840 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9844ee1-953b-4be9-97cf-6cd467d16f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:12.515730 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9844ee1-953b-4be9-97cf-6cd467d16f54 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:12.516034 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9844ee1-953b-4be9-97cf-6cd467d16f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:12.516311 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9844ee1-953b-4be9-97cf-6cd467d16f54 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:12.516970 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9844ee1-953b-4be9-97cf-6cd467d16f54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:12.518551 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924a5393-d0b0-4aec-9ec0-00d69ce4a800 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:12.521195 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9844ee1-953b-4be9-97cf-6cd467d16f54 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:12.553046 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:07:12.575396 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924a5393-d0b0-4aec-9ec0-00d69ce4a800 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:12.581196 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1117/2226] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:07:12 2022] GET /identity/v3/roles => generated 1420 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:07:12.589504 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9844ee1-953b-4be9-97cf-6cd467d16f54 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:12.600679 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc79e29-b9d8-4ab3-9688-960ec71cf4df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:12.602285 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc79e29-b9d8-4ab3-9688-960ec71cf4df admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:12.602285 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc79e29-b9d8-4ab3-9688-960ec71cf4df admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:12.602285 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc79e29-b9d8-4ab3-9688-960ec71cf4df admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:12.602992 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc79e29-b9d8-4ab3-9688-960ec71cf4df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:12.604645 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc79e29-b9d8-4ab3-9688-960ec71cf4df admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:12.622456 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9844ee1-953b-4be9-97cf-6cd467d16f54 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:12.638642 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4824151-02f8-4220-b1da-d701683f68e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:12.639335 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4824151-02f8-4220-b1da-d701683f68e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:12.639480 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1110/2227] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:07:12 2022] GET /identity/v3/roles => generated 1420 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:07:12.639667 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4824151-02f8-4220-b1da-d701683f68e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:12.639866 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4824151-02f8-4220-b1da-d701683f68e3 admin admin] PATH_INFO: `/v3/projects/da458d79154c42a7821ead0b5d2e30ae/users/7dfc57d3f7fe4089aacb85ecfdcfaf2b/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:12.643706 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc79e29-b9d8-4ab3-9688-960ec71cf4df admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:12.643888 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4824151-02f8-4220-b1da-d701683f68e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da458d79154c42a7821ead0b5d2e30ae, user_id=7dfc57d3f7fe4089aacb85ecfdcfaf2b, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:12.646668 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4824151-02f8-4220-b1da-d701683f68e3 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:12.662571 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc79e29-b9d8-4ab3-9688-960ec71cf4df admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:12.680478 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1118/2228] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 04:07:12 2022] PUT /identity/v3/projects/da458d79154c42a7821ead0b5d2e30ae/users/7dfc57d3f7fe4089aacb85ecfdcfaf2b/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:07:12.684754 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d714bd-6052-487d-aade-f82e50096546 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:12.685170 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d714bd-6052-487d-aade-f82e50096546 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:12.688288 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d714bd-6052-487d-aade-f82e50096546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:12.760712 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92e6b29-bb1d-4326-b98d-01c7c20530bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:12.761823 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92e6b29-bb1d-4326-b98d-01c7c20530bb admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:12.762279 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92e6b29-bb1d-4326-b98d-01c7c20530bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:12.762778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92e6b29-bb1d-4326-b98d-01c7c20530bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:12.763737 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92e6b29-bb1d-4326-b98d-01c7c20530bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:12.766269 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92e6b29-bb1d-4326-b98d-01c7c20530bb admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:12.774871 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-a5d714bd-6052-487d-aade-f82e50096546 None None] MFA Rules not processed for user `7dfc57d3f7fe4089aacb85ecfdcfaf2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:07:12.778858 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a5d714bd-6052-487d-aade-f82e50096546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:12.788759 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1111/2229] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:07:12 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:07:12.794123 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92e6b29-bb1d-4326-b98d-01c7c20530bb admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:12.822262 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92e6b29-bb1d-4326-b98d-01c7c20530bb admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:12.832591 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8331e1-2741-4fb5-afaa-28b9b51f3444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:12.833700 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8331e1-2741-4fb5-afaa-28b9b51f3444 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:12.833977 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8331e1-2741-4fb5-afaa-28b9b51f3444 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:12.834251 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8331e1-2741-4fb5-afaa-28b9b51f3444 admin admin] PATH_INFO: `/v3/projects/fb352bf41fa54242aea698793abbf5a4/users/78a99840af6741b689bf50fc6cddbd7c/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:12.867021 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8331e1-2741-4fb5-afaa-28b9b51f3444 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb352bf41fa54242aea698793abbf5a4, user_id=78a99840af6741b689bf50fc6cddbd7c, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:12.871033 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8331e1-2741-4fb5-afaa-28b9b51f3444 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:12.896782 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1112/2230] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 04:07:12 2022] PUT /identity/v3/projects/fb352bf41fa54242aea698793abbf5a4/users/78a99840af6741b689bf50fc6cddbd7c/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:07:12.908854 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ab3b71-5625-4f06-902a-22b650758268 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:12.909283 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ab3b71-5625-4f06-902a-22b650758268 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:12.909696 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ab3b71-5625-4f06-902a-22b650758268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:12.964985 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-03ab3b71-5625-4f06-902a-22b650758268 None None] MFA Rules not processed for user `78a99840af6741b689bf50fc6cddbd7c`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:07:12.970079 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-03ab3b71-5625-4f06-902a-22b650758268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:12.995438 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1119/2231] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:07:12 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:07:13.009754 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d714bd-6052-487d-aade-f82e50096546 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:13.011513 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a5d714bd-6052-487d-aade-f82e50096546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:13.153777 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1113/2232] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:07:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 04:07:13.163976 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d714bd-6052-487d-aade-f82e50096546 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:13.167925 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ab3b71-5625-4f06-902a-22b650758268 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:13.171028 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-03ab3b71-5625-4f06-902a-22b650758268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:13.308012 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e53bbae-248f-42d2-a1fe-4fbbb924a9dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:13.308726 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e53bbae-248f-42d2-a1fe-4fbbb924a9dc service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:13.309255 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e53bbae-248f-42d2-a1fe-4fbbb924a9dc service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:13.309538 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e53bbae-248f-42d2-a1fe-4fbbb924a9dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:13.311424 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-7e53bbae-248f-42d2-a1fe-4fbbb924a9dc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:13.318826 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:07:13.363322 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ab3b71-5625-4f06-902a-22b650758268 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:13.367741 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b009f37-5359-4eec-9d76-15fde74474e0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50aab624ee9c4651a7cd42124bcc465f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:13.368425 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b009f37-5359-4eec-9d76-15fde74474e0 service glance] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:13.368710 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b009f37-5359-4eec-9d76-15fde74474e0 service glance] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:13.368985 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b009f37-5359-4eec-9d76-15fde74474e0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:13.375449 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:07:13.388118 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b009f37-5359-4eec-9d76-15fde74474e0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:13.389577 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b009f37-5359-4eec-9d76-15fde74474e0 service glance] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:13.423991 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1114/2233] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:07:13 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:07:13.480906 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e53bbae-248f-42d2-a1fe-4fbbb924a9dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:13.482391 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e53bbae-248f-42d2-a1fe-4fbbb924a9dc service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:13.515075 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1120/2234] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:07:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 506 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 04:07:15.423927 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325fc473-b846-40cc-8060-5998c2c8def0 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:15.424483 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325fc473-b846-40cc-8060-5998c2c8def0 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:15.424732 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325fc473-b846-40cc-8060-5998c2c8def0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:15.449740 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e53bbae-248f-42d2-a1fe-4fbbb924a9dc service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:15.451399 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-7e53bbae-248f-42d2-a1fe-4fbbb924a9dc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:15.466263 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-325fc473-b846-40cc-8060-5998c2c8def0 None None] MFA Rules not processed for user `7f89c4e2fa8c495fa8bc7f64a3635fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:07:15.467568 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-325fc473-b846-40cc-8060-5998c2c8def0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:15.470762 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e53bbae-248f-42d2-a1fe-4fbbb924a9dc service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:15.475833 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8348cf8-84f0-4a53-8934-3a949ad4c2a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:15.476555 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8348cf8-84f0-4a53-8934-3a949ad4c2a9 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:15.479577 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8348cf8-84f0-4a53-8934-3a949ad4c2a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:15.479577 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8348cf8-84f0-4a53-8934-3a949ad4c2a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:15.479577 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e8348cf8-84f0-4a53-8934-3a949ad4c2a9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:15.579768 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1115/2235] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:07:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 04:07:15.583531 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325fc473-b846-40cc-8060-5998c2c8def0 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:15.585739 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-325fc473-b846-40cc-8060-5998c2c8def0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:15.596359 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8348cf8-84f0-4a53-8934-3a949ad4c2a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:15.598124 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8348cf8-84f0-4a53-8934-3a949ad4c2a9 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:15.612171 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1121/2236] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:07:15 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:07:15.701574 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325fc473-b846-40cc-8060-5998c2c8def0 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:15.710234 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9da388e-bb7b-4be2-9566-828f0be74a5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:15.711258 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9da388e-bb7b-4be2-9566-828f0be74a5e admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:15.711570 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9da388e-bb7b-4be2-9566-828f0be74a5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:15.711846 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9da388e-bb7b-4be2-9566-828f0be74a5e admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:15.712539 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9da388e-bb7b-4be2-9566-828f0be74a5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:15.715265 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9da388e-bb7b-4be2-9566-828f0be74a5e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:15.760126 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1116/2237] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 04:07:15 2022] POST /identity/v3/projects => generated 371 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:07:15.762355 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8348cf8-84f0-4a53-8934-3a949ad4c2a9 service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:15.789167 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8348cf8-84f0-4a53-8934-3a949ad4c2a9 service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:15.860674 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74daffce-9b9d-4531-9ecf-cac8409d9334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:15.861382 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74daffce-9b9d-4531-9ecf-cac8409d9334 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:15.861729 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74daffce-9b9d-4531-9ecf-cac8409d9334 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:15.862104 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74daffce-9b9d-4531-9ecf-cac8409d9334 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:15.862958 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74daffce-9b9d-4531-9ecf-cac8409d9334 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:15.865162 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74daffce-9b9d-4531-9ecf-cac8409d9334 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:15.889883 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1122/2238] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 04:07:15 2022] POST /identity/v3/users => generated 292 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:07:15.894230 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9da388e-bb7b-4be2-9566-828f0be74a5e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:15.904588 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9da388e-bb7b-4be2-9566-828f0be74a5e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:15.909397 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d831c0a-f0f7-415d-899b-b69a909f2205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:15.910016 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d831c0a-f0f7-415d-899b-b69a909f2205 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:15.910298 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d831c0a-f0f7-415d-899b-b69a909f2205 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:15.910575 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d831c0a-f0f7-415d-899b-b69a909f2205 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:15.911232 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d831c0a-f0f7-415d-899b-b69a909f2205 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:15.912862 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d831c0a-f0f7-415d-899b-b69a909f2205 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:15.923842 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1117/2239] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:07:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:07:15.928936 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74daffce-9b9d-4531-9ecf-cac8409d9334 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:15.942243 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74daffce-9b9d-4531-9ecf-cac8409d9334 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:15.948068 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d0d6e1-2e5d-4216-9535-7b2180a8b9c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:15.948780 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0d6e1-2e5d-4216-9535-7b2180a8b9c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:15.949078 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0d6e1-2e5d-4216-9535-7b2180a8b9c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:15.949334 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d0d6e1-2e5d-4216-9535-7b2180a8b9c6 admin admin] PATH_INFO: `/v3/projects/01996d414de340c39117fd2c347da57c/users/18c636cd388b4d6cbf77f9c97d3f5a76/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:15.968220 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d0d6e1-2e5d-4216-9535-7b2180a8b9c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01996d414de340c39117fd2c347da57c, user_id=18c636cd388b4d6cbf77f9c97d3f5a76, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:15.970948 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d0d6e1-2e5d-4216-9535-7b2180a8b9c6 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:15.986548 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1123/2240] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 04:07:15 2022] PUT /identity/v3/projects/01996d414de340c39117fd2c347da57c/users/18c636cd388b4d6cbf77f9c97d3f5a76/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 04:07:15.990899 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d831c0a-f0f7-415d-899b-b69a909f2205 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:16.003171 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d831c0a-f0f7-415d-899b-b69a909f2205 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:16.050115 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:07:16.073642 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2649dd-8bf1-4520-b0e3-fdb73c2de4bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:16.074456 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2649dd-8bf1-4520-b0e3-fdb73c2de4bb admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:16.074829 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2649dd-8bf1-4520-b0e3-fdb73c2de4bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:16.075099 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2649dd-8bf1-4520-b0e3-fdb73c2de4bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:16.075834 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2649dd-8bf1-4520-b0e3-fdb73c2de4bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:16.078240 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2649dd-8bf1-4520-b0e3-fdb73c2de4bb admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:16.090574 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1118/2241] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:07:15 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:07:16.095253 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d0d6e1-2e5d-4216-9535-7b2180a8b9c6 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:16.104875 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d0d6e1-2e5d-4216-9535-7b2180a8b9c6 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:16.109890 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51af0520-3d16-4940-9ca9-fcfebb8ef87d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:16.110463 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af0520-3d16-4940-9ca9-fcfebb8ef87d admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:16.110791 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af0520-3d16-4940-9ca9-fcfebb8ef87d admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:16.110996 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51af0520-3d16-4940-9ca9-fcfebb8ef87d admin admin] PATH_INFO: `/v3/projects/01996d414de340c39117fd2c347da57c/users/18c636cd388b4d6cbf77f9c97d3f5a76/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:16.113298 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51af0520-3d16-4940-9ca9-fcfebb8ef87d admin admin] RBAC: Authorizing `identity:create_grant(project_id=01996d414de340c39117fd2c347da57c, user_id=18c636cd388b4d6cbf77f9c97d3f5a76, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:16.116186 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51af0520-3d16-4940-9ca9-fcfebb8ef87d admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:16.135119 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1124/2242] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 04:07:16 2022] PUT /identity/v3/projects/01996d414de340c39117fd2c347da57c/users/18c636cd388b4d6cbf77f9c97d3f5a76/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 04:07:16.323088 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2649dd-8bf1-4520-b0e3-fdb73c2de4bb admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:16.404304 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2649dd-8bf1-4520-b0e3-fdb73c2de4bb admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:16.472328 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c3c01b-17ae-4c54-9fbc-ad4c69fc3eb1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:16.473074 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c3c01b-17ae-4c54-9fbc-ad4c69fc3eb1 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:16.473192 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c3c01b-17ae-4c54-9fbc-ad4c69fc3eb1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:16.473456 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c3c01b-17ae-4c54-9fbc-ad4c69fc3eb1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:16.564311 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c3c01b-17ae-4c54-9fbc-ad4c69fc3eb1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:16.565765 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c3c01b-17ae-4c54-9fbc-ad4c69fc3eb1 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:16.589964 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1119/2243] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:07:16 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:07:21.016857 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51af0520-3d16-4940-9ca9-fcfebb8ef87d admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:21.033514 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:07:21.135230 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51af0520-3d16-4940-9ca9-fcfebb8ef87d admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:21.155487 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abd6fed-b463-47ca-abc8-660c9be55df9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:21.156294 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd6fed-b463-47ca-abc8-660c9be55df9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:21.156668 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd6fed-b463-47ca-abc8-660c9be55df9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:21.157040 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abd6fed-b463-47ca-abc8-660c9be55df9 admin admin] PATH_INFO: `/v3/users/ddec8a4305d6441a8b19f184543d997f` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:21.158331 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abd6fed-b463-47ca-abc8-660c9be55df9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ddec8a4305d6441a8b19f184543d997f)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:21.160425 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abd6fed-b463-47ca-abc8-660c9be55df9 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:21.271399 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470cf3b-b7c3-4f57-a158-3f81d99ea715 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:21.271892 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470cf3b-b7c3-4f57-a158-3f81d99ea715 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:21.272020 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470cf3b-b7c3-4f57-a158-3f81d99ea715 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:21.290150 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-8abd6fed-b463-47ca-abc8-660c9be55df9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ddec8a4305d6441a8b19f184543d997f'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:21.290936 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-8abd6fed-b463-47ca-abc8-660c9be55df9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ddec8a4305d6441a8b19f184543d997f'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:21.291642 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-8abd6fed-b463-47ca-abc8-660c9be55df9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ddec8a4305d6441a8b19f184543d997f'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:21.314605 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-8abd6fed-b463-47ca-abc8-660c9be55df9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ddec8a4305d6441a8b19f184543d997f'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:21.332442 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-8abd6fed-b463-47ca-abc8-660c9be55df9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ddec8a4305d6441a8b19f184543d997f'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:21.338823 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-3470cf3b-b7c3-4f57-a158-3f81d99ea715 None None] MFA Rules not processed for user `18c636cd388b4d6cbf77f9c97d3f5a76`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:07:21.340735 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3470cf3b-b7c3-4f57-a158-3f81d99ea715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:21.348591 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1125/2244] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 04:07:21 2022] DELETE /identity/v3/users/ddec8a4305d6441a8b19f184543d997f => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:07:21.353635 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec24ad7d-d4f4-42bc-b839-7c29af8f8d5b None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:21.353831 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec24ad7d-d4f4-42bc-b839-7c29af8f8d5b None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:21.354106 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec24ad7d-d4f4-42bc-b839-7c29af8f8d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:21.382335 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-ec24ad7d-d4f4-42bc-b839-7c29af8f8d5b None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:07:21.391434 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ec24ad7d-d4f4-42bc-b839-7c29af8f8d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:21.400337 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:07:21.505060 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1120/2245] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:07:21 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 04:07:21.521245 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3470cf3b-b7c3-4f57-a158-3f81d99ea715 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:21.523556 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3470cf3b-b7c3-4f57-a158-3f81d99ea715 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:21.548959 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1126/2246] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:07:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:07:21.558979 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec24ad7d-d4f4-42bc-b839-7c29af8f8d5b None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:21.560788 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ec24ad7d-d4f4-42bc-b839-7c29af8f8d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:21.605691 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:07:21.630164 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3470cf3b-b7c3-4f57-a158-3f81d99ea715 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:21.638402 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec24ad7d-d4f4-42bc-b839-7c29af8f8d5b None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:21.647273 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3694ef-0a36-4970-b3ae-cccc3b4e96e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:21.647909 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3694ef-0a36-4970-b3ae-cccc3b4e96e9 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:21.648221 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3694ef-0a36-4970-b3ae-cccc3b4e96e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:21.648494 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3694ef-0a36-4970-b3ae-cccc3b4e96e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:21.658418 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3694ef-0a36-4970-b3ae-cccc3b4e96e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:21.661486 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3694ef-0a36-4970-b3ae-cccc3b4e96e9 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:21.673789 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1127/2247] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:07:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 04:07:21.715938 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6000da81-f8d1-4d13-8488-86f488795445 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:21.717119 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6000da81-f8d1-4d13-8488-86f488795445 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:21.717402 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6000da81-f8d1-4d13-8488-86f488795445 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:21.718107 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6000da81-f8d1-4d13-8488-86f488795445 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:21.720174 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6000da81-f8d1-4d13-8488-86f488795445 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:21.813303 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6000da81-f8d1-4d13-8488-86f488795445 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:21.815303 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6000da81-f8d1-4d13-8488-86f488795445 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:21.830945 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1121/2248] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:07:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:07:22.022737 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3694ef-0a36-4970-b3ae-cccc3b4e96e9 service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:22.024704 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-db3694ef-0a36-4970-b3ae-cccc3b4e96e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:22.046952 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3694ef-0a36-4970-b3ae-cccc3b4e96e9 service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:22.056290 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df51611a-c3e0-4cc9-9e93-ba6de047d717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:22.057505 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df51611a-c3e0-4cc9-9e93-ba6de047d717 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:22.057795 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df51611a-c3e0-4cc9-9e93-ba6de047d717 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:22.058403 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df51611a-c3e0-4cc9-9e93-ba6de047d717 admin admin] PATH_INFO: `/v3/projects/4261bef348d94cfc9ef05ccc990984b2` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:22.059788 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df51611a-c3e0-4cc9-9e93-ba6de047d717 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4261bef348d94cfc9ef05ccc990984b2)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:22.061761 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df51611a-c3e0-4cc9-9e93-ba6de047d717 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:22.116919 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-df51611a-c3e0-4cc9-9e93-ba6de047d717 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4261bef348d94cfc9ef05ccc990984b2', 'request_id': 'req-df51611a-c3e0-4cc9-9e93-ba6de047d717'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:22.130836 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-df51611a-c3e0-4cc9-9e93-ba6de047d717 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4261bef348d94cfc9ef05ccc990984b2', 'request_id': 'req-df51611a-c3e0-4cc9-9e93-ba6de047d717'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:22.137825 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-df51611a-c3e0-4cc9-9e93-ba6de047d717 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4261bef348d94cfc9ef05ccc990984b2', 'request_id': 'req-df51611a-c3e0-4cc9-9e93-ba6de047d717'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:22.150656 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-df51611a-c3e0-4cc9-9e93-ba6de047d717 admin admin] The token cache is being invalidate because project 4261bef348d94cfc9ef05ccc990984b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 04:07:22.150985 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-df51611a-c3e0-4cc9-9e93-ba6de047d717 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:22.151614 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-df51611a-c3e0-4cc9-9e93-ba6de047d717 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:22.153350 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1128/2249] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 04:07:22 2022] DELETE /identity/v3/projects/4261bef348d94cfc9ef05ccc990984b2 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:07:22.160296 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6000da81-f8d1-4d13-8488-86f488795445 service nova] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:22.167367 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:07:22.175320 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6000da81-f8d1-4d13-8488-86f488795445 service nova] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:22.241919 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba1e085-1f01-4aa1-a22a-0c9d3ebbd9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:22.242622 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba1e085-1f01-4aa1-a22a-0c9d3ebbd9a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:22.242907 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba1e085-1f01-4aa1-a22a-0c9d3ebbd9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:22.243196 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba1e085-1f01-4aa1-a22a-0c9d3ebbd9a8 admin admin] PATH_INFO: `/v3/users/639cc6fb33f045cdb32591db0b89751c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:22.244387 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba1e085-1f01-4aa1-a22a-0c9d3ebbd9a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=639cc6fb33f045cdb32591db0b89751c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:22.246329 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba1e085-1f01-4aa1-a22a-0c9d3ebbd9a8 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:22.300498 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-bba1e085-1f01-4aa1-a22a-0c9d3ebbd9a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '639cc6fb33f045cdb32591db0b89751c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:22.307640 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-bba1e085-1f01-4aa1-a22a-0c9d3ebbd9a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '639cc6fb33f045cdb32591db0b89751c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:22.308413 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-bba1e085-1f01-4aa1-a22a-0c9d3ebbd9a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '639cc6fb33f045cdb32591db0b89751c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:22.321832 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-bba1e085-1f01-4aa1-a22a-0c9d3ebbd9a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '639cc6fb33f045cdb32591db0b89751c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:22.330466 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-bba1e085-1f01-4aa1-a22a-0c9d3ebbd9a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '639cc6fb33f045cdb32591db0b89751c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:22.337513 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1122/2250] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 04:07:22 2022] DELETE /identity/v3/users/639cc6fb33f045cdb32591db0b89751c => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:07:22.660259 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df51611a-c3e0-4cc9-9e93-ba6de047d717 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:22.661676 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-df51611a-c3e0-4cc9-9e93-ba6de047d717 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:22.681771 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df51611a-c3e0-4cc9-9e93-ba6de047d717 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:22.750611 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db36a01-c20b-4152-a258-8b380f096438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:22.751385 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db36a01-c20b-4152-a258-8b380f096438 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:22.751860 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db36a01-c20b-4152-a258-8b380f096438 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:22.752172 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db36a01-c20b-4152-a258-8b380f096438 admin admin] PATH_INFO: `/v3/projects/551ee82723344f0e9f08f355de828bc2` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:22.753445 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db36a01-c20b-4152-a258-8b380f096438 admin admin] RBAC: Authorizing `identity:delete_project(project_id=551ee82723344f0e9f08f355de828bc2)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:22.755605 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db36a01-c20b-4152-a258-8b380f096438 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:22.809177 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1db36a01-c20b-4152-a258-8b380f096438 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '551ee82723344f0e9f08f355de828bc2', 'request_id': 'req-1db36a01-c20b-4152-a258-8b380f096438'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:22.813854 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:07:22.815335 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1db36a01-c20b-4152-a258-8b380f096438 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '551ee82723344f0e9f08f355de828bc2', 'request_id': 'req-1db36a01-c20b-4152-a258-8b380f096438'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:22.823695 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1db36a01-c20b-4152-a258-8b380f096438 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '551ee82723344f0e9f08f355de828bc2', 'request_id': 'req-1db36a01-c20b-4152-a258-8b380f096438'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:22.860936 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1db36a01-c20b-4152-a258-8b380f096438 admin admin] The token cache is being invalidate because project 551ee82723344f0e9f08f355de828bc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 04:07:22.861599 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1db36a01-c20b-4152-a258-8b380f096438 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:22.862360 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-1db36a01-c20b-4152-a258-8b380f096438 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:22.864288 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1129/2251] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 04:07:22 2022] DELETE /identity/v3/projects/551ee82723344f0e9f08f355de828bc2 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:07:22.868114 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba1e085-1f01-4aa1-a22a-0c9d3ebbd9a8 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:22.869588 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-bba1e085-1f01-4aa1-a22a-0c9d3ebbd9a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:22.889092 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba1e085-1f01-4aa1-a22a-0c9d3ebbd9a8 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:22.993740 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a00e2e-021b-4986-84e5-b0a4c5d166e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:22.994835 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a00e2e-021b-4986-84e5-b0a4c5d166e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:22.994835 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a00e2e-021b-4986-84e5-b0a4c5d166e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:22.995064 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a00e2e-021b-4986-84e5-b0a4c5d166e4 admin admin] PATH_INFO: `/v3/users/7dfc57d3f7fe4089aacb85ecfdcfaf2b` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:22.996114 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a00e2e-021b-4986-84e5-b0a4c5d166e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7dfc57d3f7fe4089aacb85ecfdcfaf2b)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:22.998048 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a00e2e-021b-4986-84e5-b0a4c5d166e4 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:23.024996 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:07:23.055143 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-93a00e2e-021b-4986-84e5-b0a4c5d166e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7dfc57d3f7fe4089aacb85ecfdcfaf2b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:23.064295 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-93a00e2e-021b-4986-84e5-b0a4c5d166e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7dfc57d3f7fe4089aacb85ecfdcfaf2b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:23.065026 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-93a00e2e-021b-4986-84e5-b0a4c5d166e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7dfc57d3f7fe4089aacb85ecfdcfaf2b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:23.079056 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-93a00e2e-021b-4986-84e5-b0a4c5d166e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7dfc57d3f7fe4089aacb85ecfdcfaf2b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:23.087621 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-93a00e2e-021b-4986-84e5-b0a4c5d166e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7dfc57d3f7fe4089aacb85ecfdcfaf2b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:23.099769 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1123/2252] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 04:07:22 2022] DELETE /identity/v3/users/7dfc57d3f7fe4089aacb85ecfdcfaf2b => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:07:23.475915 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db36a01-c20b-4152-a258-8b380f096438 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:23.477416 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-1db36a01-c20b-4152-a258-8b380f096438 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:23.495043 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db36a01-c20b-4152-a258-8b380f096438 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:23.618897 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c861a02-b36d-4754-9580-218d0d930a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:23.619932 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c861a02-b36d-4754-9580-218d0d930a24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:23.620541 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c861a02-b36d-4754-9580-218d0d930a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:23.620986 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c861a02-b36d-4754-9580-218d0d930a24 admin admin] PATH_INFO: `/v3/projects/da458d79154c42a7821ead0b5d2e30ae` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:23.622733 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c861a02-b36d-4754-9580-218d0d930a24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da458d79154c42a7821ead0b5d2e30ae)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:23.627217 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c861a02-b36d-4754-9580-218d0d930a24 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:23.704663 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2c861a02-b36d-4754-9580-218d0d930a24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da458d79154c42a7821ead0b5d2e30ae', 'request_id': 'req-2c861a02-b36d-4754-9580-218d0d930a24'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:23.712617 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2c861a02-b36d-4754-9580-218d0d930a24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da458d79154c42a7821ead0b5d2e30ae', 'request_id': 'req-2c861a02-b36d-4754-9580-218d0d930a24'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:23.723732 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2c861a02-b36d-4754-9580-218d0d930a24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da458d79154c42a7821ead0b5d2e30ae', 'request_id': 'req-2c861a02-b36d-4754-9580-218d0d930a24'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:23.744843 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2c861a02-b36d-4754-9580-218d0d930a24 admin admin] The token cache is being invalidate because project da458d79154c42a7821ead0b5d2e30ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 04:07:23.745222 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2c861a02-b36d-4754-9580-218d0d930a24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:23.745786 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2c861a02-b36d-4754-9580-218d0d930a24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:23.747309 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1130/2253] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 04:07:23 2022] DELETE /identity/v3/projects/da458d79154c42a7821ead0b5d2e30ae => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:07:23.752477 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc1640-0b66-42dd-bff5-ff7cda24bcce None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:23.753335 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc1640-0b66-42dd-bff5-ff7cda24bcce None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:23.753757 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afc1640-0b66-42dd-bff5-ff7cda24bcce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:23.790163 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-3afc1640-0b66-42dd-bff5-ff7cda24bcce None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:07:23.791714 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3afc1640-0b66-42dd-bff5-ff7cda24bcce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:23.880276 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1124/2254] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:07:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:07:23.891581 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fdeea1-63b9-4293-9352-0e6afe19f396 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:23.891857 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fdeea1-63b9-4293-9352-0e6afe19f396 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:23.893393 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fdeea1-63b9-4293-9352-0e6afe19f396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:23.941726 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-a1fdeea1-63b9-4293-9352-0e6afe19f396 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:07:23.945386 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a1fdeea1-63b9-4293-9352-0e6afe19f396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:23.973299 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1131/2255] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:07:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 04:07:23.977603 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afc1640-0b66-42dd-bff5-ff7cda24bcce None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:23.994406 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afc1640-0b66-42dd-bff5-ff7cda24bcce None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:24.016952 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b173b8be-64ca-462c-8659-291bdf12f56e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:24.017862 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b173b8be-64ca-462c-8659-291bdf12f56e admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:24.018316 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b173b8be-64ca-462c-8659-291bdf12f56e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:24.019052 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b173b8be-64ca-462c-8659-291bdf12f56e admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:24.019884 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b173b8be-64ca-462c-8659-291bdf12f56e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 04:07:24.020893 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b173b8be-64ca-462c-8659-291bdf12f56e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:24.022839 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b173b8be-64ca-462c-8659-291bdf12f56e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:24.054169 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1125/2256] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 04:07:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 04:07:24.058144 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fdeea1-63b9-4293-9352-0e6afe19f396 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:24.069441 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fdeea1-63b9-4293-9352-0e6afe19f396 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:24.075081 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de221e8-7e81-4060-8f6b-ba9df14fce5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:24.076157 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de221e8-7e81-4060-8f6b-ba9df14fce5a admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:24.076590 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de221e8-7e81-4060-8f6b-ba9df14fce5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:24.076970 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de221e8-7e81-4060-8f6b-ba9df14fce5a admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:24.078770 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de221e8-7e81-4060-8f6b-ba9df14fce5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:24.085396 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de221e8-7e81-4060-8f6b-ba9df14fce5a admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:24.140756 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1132/2257] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 04:07:24 2022] POST /identity/v3/projects => generated 371 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:07:24.145109 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b173b8be-64ca-462c-8659-291bdf12f56e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:24.161578 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b173b8be-64ca-462c-8659-291bdf12f56e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:24.245480 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcc573e-e6ba-4dea-8729-91b67ad3fb58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:24.246867 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc573e-e6ba-4dea-8729-91b67ad3fb58 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:24.247331 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc573e-e6ba-4dea-8729-91b67ad3fb58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:24.247828 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcc573e-e6ba-4dea-8729-91b67ad3fb58 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:24.249884 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcc573e-e6ba-4dea-8729-91b67ad3fb58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:24.252586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcc573e-e6ba-4dea-8729-91b67ad3fb58 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:24.284705 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1126/2258] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 04:07:24 2022] POST /identity/v3/users => generated 286 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 04:07:24.288866 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de221e8-7e81-4060-8f6b-ba9df14fce5a admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:24.302060 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dfdc4d-50dd-4446-85e7-22f2a8d68dcc None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:24.303121 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dfdc4d-50dd-4446-85e7-22f2a8d68dcc None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:24.303402 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dfdc4d-50dd-4446-85e7-22f2a8d68dcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:24.306654 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de221e8-7e81-4060-8f6b-ba9df14fce5a admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:24.315154 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959a3633-7984-4dc8-87ec-3a3a121e5d40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:24.316145 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a3633-7984-4dc8-87ec-3a3a121e5d40 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:24.319005 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a3633-7984-4dc8-87ec-3a3a121e5d40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:24.319313 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959a3633-7984-4dc8-87ec-3a3a121e5d40 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:24.320017 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959a3633-7984-4dc8-87ec-3a3a121e5d40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:24.322025 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959a3633-7984-4dc8-87ec-3a3a121e5d40 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:24.328982 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d9dfdc4d-50dd-4446-85e7-22f2a8d68dcc None None] MFA Rules not processed for user `7f89c4e2fa8c495fa8bc7f64a3635fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:07:24.329666 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d9dfdc4d-50dd-4446-85e7-22f2a8d68dcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:24.337061 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1133/2259] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:07:24 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 04:07:24.343802 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959a3633-7984-4dc8-87ec-3a3a121e5d40 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:24.357599 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:07:24.363681 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959a3633-7984-4dc8-87ec-3a3a121e5d40 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:24.381827 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05d5e45-d773-4d53-8185-0297fd6936f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:24.382602 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05d5e45-d773-4d53-8185-0297fd6936f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:24.382885 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05d5e45-d773-4d53-8185-0297fd6936f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:24.383267 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05d5e45-d773-4d53-8185-0297fd6936f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:24.383803 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05d5e45-d773-4d53-8185-0297fd6936f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:24.385435 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05d5e45-d773-4d53-8185-0297fd6936f7 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:24.404002 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1134/2260] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:07:24 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:07:24.409375 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05d5e45-d773-4d53-8185-0297fd6936f7 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:24.425090 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05d5e45-d773-4d53-8185-0297fd6936f7 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:24.430550 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff06a58-aedf-4c15-ad60-ef5af49c40a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:24.431215 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff06a58-aedf-4c15-ad60-ef5af49c40a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:24.431607 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff06a58-aedf-4c15-ad60-ef5af49c40a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:24.431884 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff06a58-aedf-4c15-ad60-ef5af49c40a4 admin admin] PATH_INFO: `/v3/projects/7b9cca9a6f634ab4833986b65af4027f/users/7c13d0e399fb427284c900bf130f23a6/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:24.436506 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1127/2261] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:07:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 04:07:24.451490 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff06a58-aedf-4c15-ad60-ef5af49c40a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b9cca9a6f634ab4833986b65af4027f, user_id=7c13d0e399fb427284c900bf130f23a6, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:24.454908 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff06a58-aedf-4c15-ad60-ef5af49c40a4 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:24.469250 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:07:24.476147 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1135/2262] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 04:07:24 2022] PUT /identity/v3/projects/7b9cca9a6f634ab4833986b65af4027f/users/7c13d0e399fb427284c900bf130f23a6/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:07:24.481207 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bcf35a-d108-4741-9e3d-f3d133af05c3 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:24.481606 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bcf35a-d108-4741-9e3d-f3d133af05c3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:24.481923 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bcf35a-d108-4741-9e3d-f3d133af05c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:24.507730 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-03bcf35a-d108-4741-9e3d-f3d133af05c3 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:07:24.509162 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-03bcf35a-d108-4741-9e3d-f3d133af05c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:24.592425 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1128/2263] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:07:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 04:07:24.602834 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff06a58-aedf-4c15-ad60-ef5af49c40a4 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:24.604982 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-eff06a58-aedf-4c15-ad60-ef5af49c40a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:24.674892 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff06a58-aedf-4c15-ad60-ef5af49c40a4 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:24.679187 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cb8faa-67b7-4fb7-ba7a-43c53b6e223d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:24.679808 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cb8faa-67b7-4fb7-ba7a-43c53b6e223d service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:24.680125 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cb8faa-67b7-4fb7-ba7a-43c53b6e223d service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:24.680408 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1cb8faa-67b7-4fb7-ba7a-43c53b6e223d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:24.692341 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cb8faa-67b7-4fb7-ba7a-43c53b6e223d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:24.693809 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1cb8faa-67b7-4fb7-ba7a-43c53b6e223d service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:24.704345 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1136/2264] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:07:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:07:25.530129 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c4f63-ead2-4eff-b7b9-723d9a63264f None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:25.530129 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c4f63-ead2-4eff-b7b9-723d9a63264f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:25.530129 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089c4f63-ead2-4eff-b7b9-723d9a63264f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:25.589019 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-089c4f63-ead2-4eff-b7b9-723d9a63264f None None] MFA Rules not processed for user `7c13d0e399fb427284c900bf130f23a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:07:25.590671 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-089c4f63-ead2-4eff-b7b9-723d9a63264f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:25.685183 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1129/2265] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:07:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:07:25.693259 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cb8faa-67b7-4fb7-ba7a-43c53b6e223d service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:25.694734 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e1cb8faa-67b7-4fb7-ba7a-43c53b6e223d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:25.777085 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1cb8faa-67b7-4fb7-ba7a-43c53b6e223d service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:25.782482 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59614828-52ce-4f33-ac6e-2e0ffec87744 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:25.783579 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59614828-52ce-4f33-ac6e-2e0ffec87744 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:25.783990 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59614828-52ce-4f33-ac6e-2e0ffec87744 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:25.784370 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59614828-52ce-4f33-ac6e-2e0ffec87744 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:25.795361 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59614828-52ce-4f33-ac6e-2e0ffec87744 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:25.797609 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59614828-52ce-4f33-ac6e-2e0ffec87744 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:25.816051 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1137/2266] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:07:25 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:07:27.244171 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49713883-fa82-43ed-aa87-9f451477ec0e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:27.244656 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49713883-fa82-43ed-aa87-9f451477ec0e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:27.244922 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49713883-fa82-43ed-aa87-9f451477ec0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:27.264108 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-49713883-fa82-43ed-aa87-9f451477ec0e None None] MFA Rules not processed for user `7c13d0e399fb427284c900bf130f23a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:07:27.265329 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-49713883-fa82-43ed-aa87-9f451477ec0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:27.272800 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1130/2267] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:07:27 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:07:35.803526 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=52 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:07:37.258677 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=79 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:07:41.865499 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59614828-52ce-4f33-ac6e-2e0ffec87744 service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:41.876948 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59614828-52ce-4f33-ac6e-2e0ffec87744 service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:41.882671 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcfdfe0-8ff5-43c8-9de0-2edadf9e193b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:41.883362 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcfdfe0-8ff5-43c8-9de0-2edadf9e193b service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:41.883731 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcfdfe0-8ff5-43c8-9de0-2edadf9e193b service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:41.884272 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcfdfe0-8ff5-43c8-9de0-2edadf9e193b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:41.886075 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ebcfdfe0-8ff5-43c8-9de0-2edadf9e193b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:41.967986 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcfdfe0-8ff5-43c8-9de0-2edadf9e193b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:41.969430 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcfdfe0-8ff5-43c8-9de0-2edadf9e193b service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:41.980295 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1138/2268] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:07:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:07:45.257877 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49713883-fa82-43ed-aa87-9f451477ec0e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:45.259790 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-49713883-fa82-43ed-aa87-9f451477ec0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:45.286449 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49713883-fa82-43ed-aa87-9f451477ec0e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:45.291379 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bf37ff-651b-4494-8b50-a686587c02a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:45.292018 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf37ff-651b-4494-8b50-a686587c02a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:45.292305 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf37ff-651b-4494-8b50-a686587c02a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:45.292577 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bf37ff-651b-4494-8b50-a686587c02a3 admin admin] PATH_INFO: `/v3/users/78a99840af6741b689bf50fc6cddbd7c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:45.295025 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bf37ff-651b-4494-8b50-a686587c02a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78a99840af6741b689bf50fc6cddbd7c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:45.298265 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bf37ff-651b-4494-8b50-a686587c02a3 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:45.378830 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-91bf37ff-651b-4494-8b50-a686587c02a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78a99840af6741b689bf50fc6cddbd7c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:45.386018 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-91bf37ff-651b-4494-8b50-a686587c02a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78a99840af6741b689bf50fc6cddbd7c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:45.386839 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-91bf37ff-651b-4494-8b50-a686587c02a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78a99840af6741b689bf50fc6cddbd7c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:45.400260 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-91bf37ff-651b-4494-8b50-a686587c02a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78a99840af6741b689bf50fc6cddbd7c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:45.409500 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-91bf37ff-651b-4494-8b50-a686587c02a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78a99840af6741b689bf50fc6cddbd7c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:45.416819 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1131/2269] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 04:07:45 2022] DELETE /identity/v3/users/78a99840af6741b689bf50fc6cddbd7c => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:07:45.421975 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f679d5-5af9-4782-b8e1-753a368f38f9 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:45.422494 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f679d5-5af9-4782-b8e1-753a368f38f9 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:45.422610 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f679d5-5af9-4782-b8e1-753a368f38f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:45.443351 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-f0f679d5-5af9-4782-b8e1-753a368f38f9 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:07:45.444705 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f0f679d5-5af9-4782-b8e1-753a368f38f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:45.511326 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1139/2270] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:07:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:07:45.521303 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bf37ff-651b-4494-8b50-a686587c02a3 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:45.522861 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-91bf37ff-651b-4494-8b50-a686587c02a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:45.584664 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bf37ff-651b-4494-8b50-a686587c02a3 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:45.588560 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcd9525-748e-4f02-a51d-1d4d18376c3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:45.589172 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcd9525-748e-4f02-a51d-1d4d18376c3b service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:45.589453 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcd9525-748e-4f02-a51d-1d4d18376c3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:45.589721 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcd9525-748e-4f02-a51d-1d4d18376c3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:45.603603 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcd9525-748e-4f02-a51d-1d4d18376c3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:45.605470 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcd9525-748e-4f02-a51d-1d4d18376c3b service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:45.616935 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1132/2271] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:07:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:07:45.886824 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f679d5-5af9-4782-b8e1-753a368f38f9 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:45.888423 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f0f679d5-5af9-4782-b8e1-753a368f38f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:45.900967 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f679d5-5af9-4782-b8e1-753a368f38f9 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:45.906147 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dac1c5d-e7d6-4a8a-816e-54d6166461ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:45.906811 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac1c5d-e7d6-4a8a-816e-54d6166461ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:45.907092 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac1c5d-e7d6-4a8a-816e-54d6166461ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:45.907339 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dac1c5d-e7d6-4a8a-816e-54d6166461ea admin admin] PATH_INFO: `/v3/projects/fb352bf41fa54242aea698793abbf5a4` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:45.908511 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dac1c5d-e7d6-4a8a-816e-54d6166461ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb352bf41fa54242aea698793abbf5a4)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:45.910427 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dac1c5d-e7d6-4a8a-816e-54d6166461ea admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:45.961972 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-9dac1c5d-e7d6-4a8a-816e-54d6166461ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb352bf41fa54242aea698793abbf5a4', 'request_id': 'req-9dac1c5d-e7d6-4a8a-816e-54d6166461ea'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:45.967470 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-9dac1c5d-e7d6-4a8a-816e-54d6166461ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb352bf41fa54242aea698793abbf5a4', 'request_id': 'req-9dac1c5d-e7d6-4a8a-816e-54d6166461ea'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:45.974203 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-9dac1c5d-e7d6-4a8a-816e-54d6166461ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb352bf41fa54242aea698793abbf5a4', 'request_id': 'req-9dac1c5d-e7d6-4a8a-816e-54d6166461ea'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:45.987607 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-9dac1c5d-e7d6-4a8a-816e-54d6166461ea admin admin] The token cache is being invalidate because project fb352bf41fa54242aea698793abbf5a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 04:07:45.987899 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-9dac1c5d-e7d6-4a8a-816e-54d6166461ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:45.988554 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-9dac1c5d-e7d6-4a8a-816e-54d6166461ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:07:45.990077 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1140/2272] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 04:07:45 2022] DELETE /identity/v3/projects/fb352bf41fa54242aea698793abbf5a4 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:07:46.410237 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a4d58-0efa-4304-a16a-d9ed8350df2f None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:46.411081 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a4d58-0efa-4304-a16a-d9ed8350df2f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:46.411354 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7a4d58-0efa-4304-a16a-d9ed8350df2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:46.430877 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-bb7a4d58-0efa-4304-a16a-d9ed8350df2f None None] MFA Rules not processed for user `7c13d0e399fb427284c900bf130f23a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:07:46.432138 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-bb7a4d58-0efa-4304-a16a-d9ed8350df2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:46.467736 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dac1c5d-e7d6-4a8a-816e-54d6166461ea admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:46.469210 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-9dac1c5d-e7d6-4a8a-816e-54d6166461ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:46.480057 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:07:46.492648 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1133/2273] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:07:46 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 04:07:46.530134 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dac1c5d-e7d6-4a8a-816e-54d6166461ea admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:46.534443 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef0497f-ffaa-498c-9adf-bcf3756c295a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:46.535065 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef0497f-ffaa-498c-9adf-bcf3756c295a service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:46.535343 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef0497f-ffaa-498c-9adf-bcf3756c295a service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:46.535712 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef0497f-ffaa-498c-9adf-bcf3756c295a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:46.537389 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-6ef0497f-ffaa-498c-9adf-bcf3756c295a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:46.548261 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef0497f-ffaa-498c-9adf-bcf3756c295a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:46.549852 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef0497f-ffaa-498c-9adf-bcf3756c295a service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:46.558398 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1141/2274] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:07:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 04:07:55.244091 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7a4d58-0efa-4304-a16a-d9ed8350df2f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:07:55.245471 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-bb7a4d58-0efa-4304-a16a-d9ed8350df2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:55.267876 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:07:55.316205 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7a4d58-0efa-4304-a16a-d9ed8350df2f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:07:55.321166 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44d4a7f-6cf4-44a5-94ed-278a252f7df7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4b9bd741574ddabadd4b7651ebc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:07:55.321777 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d4a7f-6cf4-44a5-94ed-278a252f7df7 service placement] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:07:55.322057 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d4a7f-6cf4-44a5-94ed-278a252f7df7 service placement] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:07:55.322331 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44d4a7f-6cf4-44a5-94ed-278a252f7df7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:07:55.324075 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b44d4a7f-6cf4-44a5-94ed-278a252f7df7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:07:55.336223 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44d4a7f-6cf4-44a5-94ed-278a252f7df7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:07:55.337803 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44d4a7f-6cf4-44a5-94ed-278a252f7df7 service placement] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:07:55.348282 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1134/2275] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:07:55 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:07:56.554375 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=22 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:08:04.725274 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef0497f-ffaa-498c-9adf-bcf3756c295a service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:08:04.734439 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef0497f-ffaa-498c-9adf-bcf3756c295a service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:08:04.738125 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853f820e-77d5-4ddd-ab42-76c6d33fbcfa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:08:04.738720 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f820e-77d5-4ddd-ab42-76c6d33fbcfa service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:08:04.739024 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f820e-77d5-4ddd-ab42-76c6d33fbcfa service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:08:04.739409 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853f820e-77d5-4ddd-ab42-76c6d33fbcfa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:08:04.741120 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-853f820e-77d5-4ddd-ab42-76c6d33fbcfa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:08:04.802502 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853f820e-77d5-4ddd-ab42-76c6d33fbcfa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:08:04.803902 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-853f820e-77d5-4ddd-ab42-76c6d33fbcfa service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:08:04.813231 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1142/2276] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:08:04 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:08:05.344028 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=23 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:08:10.370434 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47315820-5da2-49e5-a95d-5bd0529a2b40 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:08:10.370977 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47315820-5da2-49e5-a95d-5bd0529a2b40 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:08:10.370977 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47315820-5da2-49e5-a95d-5bd0529a2b40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:08:10.430869 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-47315820-5da2-49e5-a95d-5bd0529a2b40 None None] MFA Rules not processed for user `da7ee607b66e43f3823d9d2faf373974`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:08:10.432098 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-47315820-5da2-49e5-a95d-5bd0529a2b40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:08:10.497540 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1135/2277] 10.222.0.26 () {58 vars in 1009 bytes} [Wed Nov 2 04:08:10 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:08:10.551367 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853f820e-77d5-4ddd-ab42-76c6d33fbcfa service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:08:10.552737 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-853f820e-77d5-4ddd-ab42-76c6d33fbcfa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:08:10.641635 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853f820e-77d5-4ddd-ab42-76c6d33fbcfa service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:08:10.646596 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962e5f56-198d-4ef8-8801-b5fc9d9befb6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5193c40a917b4105ab8b27aead1685f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:08:10.647192 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e5f56-198d-4ef8-8801-b5fc9d9befb6 service swift] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:08:10.647467 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e5f56-198d-4ef8-8801-b5fc9d9befb6 service swift] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:08:10.647734 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e5f56-198d-4ef8-8801-b5fc9d9befb6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:08:10.656141 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962e5f56-198d-4ef8-8801-b5fc9d9befb6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:08:10.657706 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962e5f56-198d-4ef8-8801-b5fc9d9befb6 service swift] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:08:10.665582 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1143/2278] 10.222.0.26 () {60 vars in 1384 bytes} [Wed Nov 2 04:08:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 04:08:20.494780 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=44 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:08:20.663699 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=71 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:08:25.559935 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbde1e0-481f-40e7-b30c-4546aaad8bc4 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:08:25.560407 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbde1e0-481f-40e7-b30c-4546aaad8bc4 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:08:25.560757 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dbde1e0-481f-40e7-b30c-4546aaad8bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:08:25.581787 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-7dbde1e0-481f-40e7-b30c-4546aaad8bc4 None None] MFA Rules not processed for user `da7ee607b66e43f3823d9d2faf373974`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:08:25.582978 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7dbde1e0-481f-40e7-b30c-4546aaad8bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:08:25.590029 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1136/2279] 10.222.0.26 () {58 vars in 967 bytes} [Wed Nov 2 04:08:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:08:25.603231 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962e5f56-198d-4ef8-8801-b5fc9d9befb6 service swift] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:08:25.613586 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962e5f56-198d-4ef8-8801-b5fc9d9befb6 service swift] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:08:25.618492 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8892cfd1-fbdd-42fa-8d82-88642053d426 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5193c40a917b4105ab8b27aead1685f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:08:25.619079 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8892cfd1-fbdd-42fa-8d82-88642053d426 service swift] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:08:25.619353 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8892cfd1-fbdd-42fa-8d82-88642053d426 service swift] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:08:25.619622 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8892cfd1-fbdd-42fa-8d82-88642053d426 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:08:25.628304 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8892cfd1-fbdd-42fa-8d82-88642053d426 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:08:25.629843 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8892cfd1-fbdd-42fa-8d82-88642053d426 service swift] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:08:25.637613 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1144/2280] 10.222.0.26 () {60 vars in 1384 bytes} [Wed Nov 2 04:08:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 04:08:27.527224 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbde1e0-481f-40e7-b30c-4546aaad8bc4 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:08:27.529279 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7dbde1e0-481f-40e7-b30c-4546aaad8bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:08:27.599546 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dbde1e0-481f-40e7-b30c-4546aaad8bc4 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:08:27.603911 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9a2e5f-a224-4c41-83dc-e96d0b650d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:08:27.604563 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a2e5f-a224-4c41-83dc-e96d0b650d05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:08:27.604839 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a2e5f-a224-4c41-83dc-e96d0b650d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:08:27.605107 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9a2e5f-a224-4c41-83dc-e96d0b650d05 admin admin] PATH_INFO: `/v3/users/7c13d0e399fb427284c900bf130f23a6` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:08:27.606215 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9a2e5f-a224-4c41-83dc-e96d0b650d05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c13d0e399fb427284c900bf130f23a6)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:08:27.608644 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a9a2e5f-a224-4c41-83dc-e96d0b650d05 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:08:27.659451 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4a9a2e5f-a224-4c41-83dc-e96d0b650d05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c13d0e399fb427284c900bf130f23a6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:08:27.666819 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4a9a2e5f-a224-4c41-83dc-e96d0b650d05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c13d0e399fb427284c900bf130f23a6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:08:27.667604 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4a9a2e5f-a224-4c41-83dc-e96d0b650d05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c13d0e399fb427284c900bf130f23a6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:08:27.680130 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4a9a2e5f-a224-4c41-83dc-e96d0b650d05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c13d0e399fb427284c900bf130f23a6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:08:27.689301 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4a9a2e5f-a224-4c41-83dc-e96d0b650d05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c13d0e399fb427284c900bf130f23a6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:08:27.695319 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1137/2281] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 04:08:27 2022] DELETE /identity/v3/users/7c13d0e399fb427284c900bf130f23a6 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:08:27.864424 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8892cfd1-fbdd-42fa-8d82-88642053d426 service swift] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:08:27.875308 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8892cfd1-fbdd-42fa-8d82-88642053d426 service swift] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:08:27.931478 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d76798-7f8c-42d1-8627-a7a0ddcbae6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:08:27.932139 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d76798-7f8c-42d1-8627-a7a0ddcbae6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:08:27.932422 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d76798-7f8c-42d1-8627-a7a0ddcbae6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:08:27.932701 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d76798-7f8c-42d1-8627-a7a0ddcbae6f admin admin] PATH_INFO: `/v3/projects/7b9cca9a6f634ab4833986b65af4027f` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:08:27.933832 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d76798-7f8c-42d1-8627-a7a0ddcbae6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b9cca9a6f634ab4833986b65af4027f)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:08:27.935735 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d76798-7f8c-42d1-8627-a7a0ddcbae6f admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:08:27.981030 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-90d76798-7f8c-42d1-8627-a7a0ddcbae6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b9cca9a6f634ab4833986b65af4027f', 'request_id': 'req-90d76798-7f8c-42d1-8627-a7a0ddcbae6f'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:08:27.987145 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-90d76798-7f8c-42d1-8627-a7a0ddcbae6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b9cca9a6f634ab4833986b65af4027f', 'request_id': 'req-90d76798-7f8c-42d1-8627-a7a0ddcbae6f'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:08:27.993249 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-90d76798-7f8c-42d1-8627-a7a0ddcbae6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b9cca9a6f634ab4833986b65af4027f', 'request_id': 'req-90d76798-7f8c-42d1-8627-a7a0ddcbae6f'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:08:28.005352 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-90d76798-7f8c-42d1-8627-a7a0ddcbae6f admin admin] The token cache is being invalidate because project 7b9cca9a6f634ab4833986b65af4027f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 04:08:28.005638 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-90d76798-7f8c-42d1-8627-a7a0ddcbae6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:08:28.006251 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-90d76798-7f8c-42d1-8627-a7a0ddcbae6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:08:28.007731 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1145/2282] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 04:08:27 2022] DELETE /identity/v3/projects/7b9cca9a6f634ab4833986b65af4027f => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:08:28.012686 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eef1b7-d4f5-42bc-bc0c-05357e25d7a1 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:08:28.013054 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eef1b7-d4f5-42bc-bc0c-05357e25d7a1 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:08:28.013395 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eef1b7-d4f5-42bc-bc0c-05357e25d7a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:08:28.033661 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-17eef1b7-d4f5-42bc-bc0c-05357e25d7a1 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:08:28.034864 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-17eef1b7-d4f5-42bc-bc0c-05357e25d7a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:08:28.093002 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1138/2283] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:08:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:08:28.100839 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac78ea-6c39-4ab5-b6e6-43a20e0253a7 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:08:28.101122 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac78ea-6c39-4ab5-b6e6-43a20e0253a7 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:08:28.101392 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac78ea-6c39-4ab5-b6e6-43a20e0253a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:08:28.121651 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-22ac78ea-6c39-4ab5-b6e6-43a20e0253a7 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:08:28.122809 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-22ac78ea-6c39-4ab5-b6e6-43a20e0253a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:08:28.130997 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1146/2284] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:08:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:08:28.135077 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eef1b7-d4f5-42bc-bc0c-05357e25d7a1 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:08:28.147021 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eef1b7-d4f5-42bc-bc0c-05357e25d7a1 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:08:28.151684 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5d31b1-02b1-44ee-9703-f5a18ed07f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:08:28.152320 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d31b1-02b1-44ee-9703-f5a18ed07f79 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:08:28.152604 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d31b1-02b1-44ee-9703-f5a18ed07f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:08:28.152881 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5d31b1-02b1-44ee-9703-f5a18ed07f79 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:08:28.153536 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5d31b1-02b1-44ee-9703-f5a18ed07f79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 04:08:28.153842 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5d31b1-02b1-44ee-9703-f5a18ed07f79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:08:28.156112 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5d31b1-02b1-44ee-9703-f5a18ed07f79 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:08:28.164856 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:08:28.168428 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1139/2285] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 04:08:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 04:08:28.172487 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ac78ea-6c39-4ab5-b6e6-43a20e0253a7 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:08:28.181710 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ac78ea-6c39-4ab5-b6e6-43a20e0253a7 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:08:28.186436 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e498402-2c10-4f48-88ed-43e56a98765b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:08:28.187045 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e498402-2c10-4f48-88ed-43e56a98765b admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:08:28.187325 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e498402-2c10-4f48-88ed-43e56a98765b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:08:28.187601 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e498402-2c10-4f48-88ed-43e56a98765b admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:08:28.188280 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e498402-2c10-4f48-88ed-43e56a98765b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:08:28.190170 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e498402-2c10-4f48-88ed-43e56a98765b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:08:28.217440 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1147/2286] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 04:08:28 2022] POST /identity/v3/projects => generated 363 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 04:08:28.222120 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5d31b1-02b1-44ee-9703-f5a18ed07f79 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:08:28.231893 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5d31b1-02b1-44ee-9703-f5a18ed07f79 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:08:28.287671 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8bd145-0168-4fe1-a8c2-9b30fc0e24c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:08:28.288321 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8bd145-0168-4fe1-a8c2-9b30fc0e24c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:08:28.288605 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8bd145-0168-4fe1-a8c2-9b30fc0e24c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:08:28.288882 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8bd145-0168-4fe1-a8c2-9b30fc0e24c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:08:28.289583 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8bd145-0168-4fe1-a8c2-9b30fc0e24c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:08:28.291531 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8bd145-0168-4fe1-a8c2-9b30fc0e24c6 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:08:28.312301 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1140/2287] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 04:08:28 2022] POST /identity/v3/users => generated 282 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:08:28.316378 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e498402-2c10-4f48-88ed-43e56a98765b admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:08:28.326533 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e498402-2c10-4f48-88ed-43e56a98765b admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:08:28.331640 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da56ffe-9968-4b85-9702-807ce3ba33c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:08:28.332273 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da56ffe-9968-4b85-9702-807ce3ba33c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:08:28.332550 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da56ffe-9968-4b85-9702-807ce3ba33c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:08:28.332823 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da56ffe-9968-4b85-9702-807ce3ba33c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:08:28.333460 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da56ffe-9968-4b85-9702-807ce3ba33c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:08:28.335072 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da56ffe-9968-4b85-9702-807ce3ba33c1 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:08:28.345849 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1148/2288] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:08:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:08:28.349753 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8bd145-0168-4fe1-a8c2-9b30fc0e24c6 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:08:28.359704 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8bd145-0168-4fe1-a8c2-9b30fc0e24c6 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:08:28.364351 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c615e1de-7fc9-42a7-801e-709733e0b98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:08:28.364950 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c615e1de-7fc9-42a7-801e-709733e0b98b admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:08:28.365232 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c615e1de-7fc9-42a7-801e-709733e0b98b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:08:28.365505 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c615e1de-7fc9-42a7-801e-709733e0b98b admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:08:28.366142 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c615e1de-7fc9-42a7-801e-709733e0b98b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:08:28.367739 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c615e1de-7fc9-42a7-801e-709733e0b98b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:08:28.378849 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1141/2289] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:08:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:08:28.382840 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da56ffe-9968-4b85-9702-807ce3ba33c1 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:08:28.393281 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da56ffe-9968-4b85-9702-807ce3ba33c1 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:08:28.397909 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47269921-e79f-4bc9-92c1-205fde58cee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:08:28.398548 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47269921-e79f-4bc9-92c1-205fde58cee3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:08:28.398826 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47269921-e79f-4bc9-92c1-205fde58cee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:08:28.399099 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47269921-e79f-4bc9-92c1-205fde58cee3 admin admin] PATH_INFO: `/v3/projects/620eaf44152d4d83a6487de48ec89ed8/users/6405f04c047d4068ab5500ed8a906950/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:08:28.414099 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47269921-e79f-4bc9-92c1-205fde58cee3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=620eaf44152d4d83a6487de48ec89ed8, user_id=6405f04c047d4068ab5500ed8a906950, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:08:28.416826 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47269921-e79f-4bc9-92c1-205fde58cee3 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:08:28.429933 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1149/2290] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 04:08:28 2022] PUT /identity/v3/projects/620eaf44152d4d83a6487de48ec89ed8/users/6405f04c047d4068ab5500ed8a906950/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:08:28.441124 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b39e577-dd2f-46de-b58c-8485a88612ab None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:08:28.441414 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b39e577-dd2f-46de-b58c-8485a88612ab None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:08:28.441690 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b39e577-dd2f-46de-b58c-8485a88612ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:08:28.470720 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-6b39e577-dd2f-46de-b58c-8485a88612ab None None] MFA Rules not processed for user `6405f04c047d4068ab5500ed8a906950`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:08:28.471927 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6b39e577-dd2f-46de-b58c-8485a88612ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:08:28.535577 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1142/2291] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:08:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:08:28.546956 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47269921-e79f-4bc9-92c1-205fde58cee3 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:08:28.548691 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-47269921-e79f-4bc9-92c1-205fde58cee3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:08:28.563034 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:08:28.606706 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47269921-e79f-4bc9-92c1-205fde58cee3 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:08:28.610935 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18ef0f2-1597-430a-a271-5de0161ab4a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:08:28.611532 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ef0f2-1597-430a-a271-5de0161ab4a0 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:08:28.611814 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ef0f2-1597-430a-a271-5de0161ab4a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:08:28.612123 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ef0f2-1597-430a-a271-5de0161ab4a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:08:28.620313 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18ef0f2-1597-430a-a271-5de0161ab4a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:08:28.621738 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18ef0f2-1597-430a-a271-5de0161ab4a0 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:08:28.630842 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1150/2292] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:08:28 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:08:38.532439 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=66,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:08:38.626810 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=20 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:09:03.648634 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b39e577-dd2f-46de-b58c-8485a88612ab None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:09:03.709972 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b39e577-dd2f-46de-b58c-8485a88612ab None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:09:03.714474 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fe08d5-e756-4c4b-893d-3154e34e0b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:09:03.715096 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fe08d5-e756-4c4b-893d-3154e34e0b28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:09:03.715375 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fe08d5-e756-4c4b-893d-3154e34e0b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:09:03.715648 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fe08d5-e756-4c4b-893d-3154e34e0b28 admin admin] PATH_INFO: `/v3/users/6405f04c047d4068ab5500ed8a906950` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:09:03.716773 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fe08d5-e756-4c4b-893d-3154e34e0b28 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6405f04c047d4068ab5500ed8a906950)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:09:03.718715 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fe08d5-e756-4c4b-893d-3154e34e0b28 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:09:03.763283 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-64fe08d5-e756-4c4b-893d-3154e34e0b28 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6405f04c047d4068ab5500ed8a906950'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:09:03.770065 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-64fe08d5-e756-4c4b-893d-3154e34e0b28 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6405f04c047d4068ab5500ed8a906950'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:09:03.770717 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-64fe08d5-e756-4c4b-893d-3154e34e0b28 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6405f04c047d4068ab5500ed8a906950'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:09:03.782346 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-64fe08d5-e756-4c4b-893d-3154e34e0b28 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6405f04c047d4068ab5500ed8a906950'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:09:03.789944 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-64fe08d5-e756-4c4b-893d-3154e34e0b28 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6405f04c047d4068ab5500ed8a906950'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:09:03.795683 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1143/2293] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 04:09:03 2022] DELETE /identity/v3/users/6405f04c047d4068ab5500ed8a906950 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:09:03.801067 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055efb23-6a3d-4a0c-82f2-20c355a5c3e4 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:09:03.801576 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055efb23-6a3d-4a0c-82f2-20c355a5c3e4 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:09:03.801704 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055efb23-6a3d-4a0c-82f2-20c355a5c3e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:09:03.820208 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-055efb23-6a3d-4a0c-82f2-20c355a5c3e4 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:09:03.821419 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-055efb23-6a3d-4a0c-82f2-20c355a5c3e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:09:03.877824 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1151/2294] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:09:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:09:03.887286 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fe08d5-e756-4c4b-893d-3154e34e0b28 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:09:03.888709 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-64fe08d5-e756-4c4b-893d-3154e34e0b28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:09:03.946608 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fe08d5-e756-4c4b-893d-3154e34e0b28 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:09:03.950396 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd1ff42-e523-4d0f-bdc7-ba79f7a55e56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:09:03.950996 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd1ff42-e523-4d0f-bdc7-ba79f7a55e56 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:09:03.951270 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd1ff42-e523-4d0f-bdc7-ba79f7a55e56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:09:03.951539 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd1ff42-e523-4d0f-bdc7-ba79f7a55e56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:09:03.960542 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd1ff42-e523-4d0f-bdc7-ba79f7a55e56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:09:03.961957 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd1ff42-e523-4d0f-bdc7-ba79f7a55e56 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:09:03.971653 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1144/2295] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:09:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:09:04.245910 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055efb23-6a3d-4a0c-82f2-20c355a5c3e4 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:09:04.247491 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-055efb23-6a3d-4a0c-82f2-20c355a5c3e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:09:04.259548 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055efb23-6a3d-4a0c-82f2-20c355a5c3e4 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:09:04.264282 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24716174-0f75-4bf3-bf99-46aeb5a9145c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:09:04.264913 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24716174-0f75-4bf3-bf99-46aeb5a9145c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:09:04.265192 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24716174-0f75-4bf3-bf99-46aeb5a9145c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:09:04.265459 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24716174-0f75-4bf3-bf99-46aeb5a9145c admin admin] PATH_INFO: `/v3/projects/620eaf44152d4d83a6487de48ec89ed8` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:09:04.266702 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24716174-0f75-4bf3-bf99-46aeb5a9145c admin admin] RBAC: Authorizing `identity:delete_project(project_id=620eaf44152d4d83a6487de48ec89ed8)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:09:04.269162 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24716174-0f75-4bf3-bf99-46aeb5a9145c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:09:04.317245 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-24716174-0f75-4bf3-bf99-46aeb5a9145c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '620eaf44152d4d83a6487de48ec89ed8', 'request_id': 'req-24716174-0f75-4bf3-bf99-46aeb5a9145c'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:09:04.322689 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-24716174-0f75-4bf3-bf99-46aeb5a9145c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '620eaf44152d4d83a6487de48ec89ed8', 'request_id': 'req-24716174-0f75-4bf3-bf99-46aeb5a9145c'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:09:04.328802 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-24716174-0f75-4bf3-bf99-46aeb5a9145c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '620eaf44152d4d83a6487de48ec89ed8', 'request_id': 'req-24716174-0f75-4bf3-bf99-46aeb5a9145c'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:09:04.342121 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-24716174-0f75-4bf3-bf99-46aeb5a9145c admin admin] The token cache is being invalidate because project 620eaf44152d4d83a6487de48ec89ed8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 04:09:04.342412 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-24716174-0f75-4bf3-bf99-46aeb5a9145c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:09:04.343047 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-24716174-0f75-4bf3-bf99-46aeb5a9145c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:09:04.344561 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1152/2296] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 04:09:04 2022] DELETE /identity/v3/projects/620eaf44152d4d83a6487de48ec89ed8 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:09:04.350354 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f99f6c-2b9d-49ce-bf9d-dc4603df1399 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:09:04.350642 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f99f6c-2b9d-49ce-bf9d-dc4603df1399 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:09:04.350915 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f99f6c-2b9d-49ce-bf9d-dc4603df1399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:09:04.370953 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-b2f99f6c-2b9d-49ce-bf9d-dc4603df1399 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:09:04.372187 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b2f99f6c-2b9d-49ce-bf9d-dc4603df1399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:09:04.389236 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:09:04.435283 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1145/2297] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:09:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:09:04.443330 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7cd947-ee32-40e4-a6fa-e60ad8adc2f8 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:09:04.443687 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7cd947-ee32-40e4-a6fa-e60ad8adc2f8 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:09:04.444131 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7cd947-ee32-40e4-a6fa-e60ad8adc2f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:09:04.464460 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-cd7cd947-ee32-40e4-a6fa-e60ad8adc2f8 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:09:04.465694 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-cd7cd947-ee32-40e4-a6fa-e60ad8adc2f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:09:04.474173 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1153/2298] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:09:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 04:09:04.478362 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f99f6c-2b9d-49ce-bf9d-dc4603df1399 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:09:04.488598 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f99f6c-2b9d-49ce-bf9d-dc4603df1399 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:09:04.493444 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1a6ae9-04ee-4b19-90a2-31c984c76a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:09:04.494051 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a6ae9-04ee-4b19-90a2-31c984c76a07 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:09:04.494326 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a6ae9-04ee-4b19-90a2-31c984c76a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:09:04.494594 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a6ae9-04ee-4b19-90a2-31c984c76a07 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:09:04.495243 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1a6ae9-04ee-4b19-90a2-31c984c76a07 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 04:09:04.495568 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1a6ae9-04ee-4b19-90a2-31c984c76a07 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:09:04.497205 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1a6ae9-04ee-4b19-90a2-31c984c76a07 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:09:04.509258 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1146/2299] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 04:09:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 04:09:04.513714 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7cd947-ee32-40e4-a6fa-e60ad8adc2f8 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:09:04.522828 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7cd947-ee32-40e4-a6fa-e60ad8adc2f8 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:09:04.527712 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b06dcf5-a51a-4afe-b517-d63bf34e2af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:09:04.528362 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b06dcf5-a51a-4afe-b517-d63bf34e2af8 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:09:04.528643 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b06dcf5-a51a-4afe-b517-d63bf34e2af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:09:04.528919 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b06dcf5-a51a-4afe-b517-d63bf34e2af8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:09:04.529561 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b06dcf5-a51a-4afe-b517-d63bf34e2af8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:09:04.531497 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b06dcf5-a51a-4afe-b517-d63bf34e2af8 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:09:04.589643 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1154/2300] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 04:09:04 2022] POST /identity/v3/projects => generated 383 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:09:04.591224 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1a6ae9-04ee-4b19-90a2-31c984c76a07 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:09:04.601914 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1a6ae9-04ee-4b19-90a2-31c984c76a07 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:09:04.659674 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ce5a1b-6809-40a5-88af-6c3f19e2edf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:09:04.660366 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce5a1b-6809-40a5-88af-6c3f19e2edf9 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:09:04.660581 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce5a1b-6809-40a5-88af-6c3f19e2edf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:09:04.660858 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ce5a1b-6809-40a5-88af-6c3f19e2edf9 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:09:04.661495 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ce5a1b-6809-40a5-88af-6c3f19e2edf9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:09:04.663398 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ce5a1b-6809-40a5-88af-6c3f19e2edf9 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:09:04.685620 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1147/2301] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 04:09:04 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:09:04.689591 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b06dcf5-a51a-4afe-b517-d63bf34e2af8 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:09:04.699774 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b06dcf5-a51a-4afe-b517-d63bf34e2af8 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:09:04.704601 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ee4f42-1541-4270-9aaa-c8e6eb4f0c29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:09:04.705207 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ee4f42-1541-4270-9aaa-c8e6eb4f0c29 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:09:04.705507 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ee4f42-1541-4270-9aaa-c8e6eb4f0c29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:09:04.705785 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ee4f42-1541-4270-9aaa-c8e6eb4f0c29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:09:04.706427 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ee4f42-1541-4270-9aaa-c8e6eb4f0c29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:09:04.708078 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ee4f42-1541-4270-9aaa-c8e6eb4f0c29 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:09:04.718162 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1155/2302] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:09:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:09:04.722879 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ce5a1b-6809-40a5-88af-6c3f19e2edf9 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:09:04.732911 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ce5a1b-6809-40a5-88af-6c3f19e2edf9 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:09:04.738214 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d63c8a7-a153-42e6-bd0b-cdedd96daf91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:09:04.738816 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63c8a7-a153-42e6-bd0b-cdedd96daf91 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:09:04.739094 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63c8a7-a153-42e6-bd0b-cdedd96daf91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:09:04.739365 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d63c8a7-a153-42e6-bd0b-cdedd96daf91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:09:04.740030 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d63c8a7-a153-42e6-bd0b-cdedd96daf91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:09:04.741631 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d63c8a7-a153-42e6-bd0b-cdedd96daf91 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:09:04.752476 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1148/2303] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:09:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:09:04.756756 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ee4f42-1541-4270-9aaa-c8e6eb4f0c29 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:09:04.766957 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ee4f42-1541-4270-9aaa-c8e6eb4f0c29 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:09:04.772273 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ca0501-16b2-4cc2-aba3-eddb8fe0ff50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:09:04.772932 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ca0501-16b2-4cc2-aba3-eddb8fe0ff50 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:09:04.773217 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ca0501-16b2-4cc2-aba3-eddb8fe0ff50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:09:04.773492 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ca0501-16b2-4cc2-aba3-eddb8fe0ff50 admin admin] PATH_INFO: `/v3/projects/561a9720212b46e2bb93367218e0f112/users/8b56999a02524408b74371e6f8d158cf/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:09:04.790395 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ca0501-16b2-4cc2-aba3-eddb8fe0ff50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=561a9720212b46e2bb93367218e0f112, user_id=8b56999a02524408b74371e6f8d158cf, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:09:04.793108 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ca0501-16b2-4cc2-aba3-eddb8fe0ff50 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:09:04.807478 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1156/2304] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 04:09:04 2022] PUT /identity/v3/projects/561a9720212b46e2bb93367218e0f112/users/8b56999a02524408b74371e6f8d158cf/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:09:04.812483 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa569137-f4e2-4491-af64-094c7cf490d7 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:09:04.812772 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa569137-f4e2-4491-af64-094c7cf490d7 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:09:04.813044 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa569137-f4e2-4491-af64-094c7cf490d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:09:04.832805 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-fa569137-f4e2-4491-af64-094c7cf490d7 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:09:04.834047 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-fa569137-f4e2-4491-af64-094c7cf490d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:09:04.893847 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1149/2305] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:09:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:09:04.903437 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ca0501-16b2-4cc2-aba3-eddb8fe0ff50 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:09:04.905197 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a8ca0501-16b2-4cc2-aba3-eddb8fe0ff50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:09:04.919115 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:09:04.964514 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ca0501-16b2-4cc2-aba3-eddb8fe0ff50 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:09:04.968166 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea185dd4-9514-411f-8b53-2151fb0f6eab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:09:04.968776 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea185dd4-9514-411f-8b53-2151fb0f6eab service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:09:04.969052 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea185dd4-9514-411f-8b53-2151fb0f6eab service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:09:04.969322 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea185dd4-9514-411f-8b53-2151fb0f6eab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:09:04.978984 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea185dd4-9514-411f-8b53-2151fb0f6eab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:09:04.980450 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea185dd4-9514-411f-8b53-2151fb0f6eab service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:09:04.989952 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1157/2306] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:09:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:09:09.021860 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf83d060-fccb-46c0-bceb-094243fd7dba None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:09:09.022311 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf83d060-fccb-46c0-bceb-094243fd7dba None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:09:09.022430 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf83d060-fccb-46c0-bceb-094243fd7dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:09:09.053309 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-bf83d060-fccb-46c0-bceb-094243fd7dba None None] MFA Rules not processed for user `8b56999a02524408b74371e6f8d158cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:09:09.054611 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-bf83d060-fccb-46c0-bceb-094243fd7dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:09:09.073109 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:09:09.132679 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1150/2307] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:09:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:09:09.137020 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea185dd4-9514-411f-8b53-2151fb0f6eab service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:09:09.153889 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea185dd4-9514-411f-8b53-2151fb0f6eab service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:09:09.159515 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f1a512-7422-4f0c-ac46-831ad2d26c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:09:09.160159 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f1a512-7422-4f0c-ac46-831ad2d26c9c admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:09:09.160439 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f1a512-7422-4f0c-ac46-831ad2d26c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:09:09.160713 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f1a512-7422-4f0c-ac46-831ad2d26c9c admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:09:09.161375 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f1a512-7422-4f0c-ac46-831ad2d26c9c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:09:09.163264 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f1a512-7422-4f0c-ac46-831ad2d26c9c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:09:09.195736 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1158/2308] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 04:09:09 2022] POST /identity/v3/projects => generated 385 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:09:09.204887 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf83d060-fccb-46c0-bceb-094243fd7dba None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:09:09.216190 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf83d060-fccb-46c0-bceb-094243fd7dba None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:09:09.279234 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9e9057-d400-4115-bd9a-cb8253f009e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:09:09.280086 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9e9057-d400-4115-bd9a-cb8253f009e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:09:09.280568 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9e9057-d400-4115-bd9a-cb8253f009e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:09:09.280936 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9e9057-d400-4115-bd9a-cb8253f009e1 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:09:09.281672 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9e9057-d400-4115-bd9a-cb8253f009e1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:09:09.284685 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9e9057-d400-4115-bd9a-cb8253f009e1 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:09:09.311998 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1151/2309] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 04:09:09 2022] POST /identity/v3/users => generated 299 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:09:09.314688 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f1a512-7422-4f0c-ac46-831ad2d26c9c admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:09:09.326133 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f1a512-7422-4f0c-ac46-831ad2d26c9c admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:09:09.333797 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd14f767-404f-4d03-9eef-035c165df71f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:09:09.334420 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd14f767-404f-4d03-9eef-035c165df71f admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:09:09.334700 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd14f767-404f-4d03-9eef-035c165df71f admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:09:09.334975 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd14f767-404f-4d03-9eef-035c165df71f admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:09:09.335626 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd14f767-404f-4d03-9eef-035c165df71f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:09:09.337321 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd14f767-404f-4d03-9eef-035c165df71f admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:09:09.349471 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1159/2310] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:09:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:09:09.357110 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9e9057-d400-4115-bd9a-cb8253f009e1 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:09:09.367615 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9e9057-d400-4115-bd9a-cb8253f009e1 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:09:09.372826 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fe1bb3-5d5d-4f81-a354-e5a94cfcbb61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:09:09.374839 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe1bb3-5d5d-4f81-a354-e5a94cfcbb61 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:09:09.375142 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe1bb3-5d5d-4f81-a354-e5a94cfcbb61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:09:09.375491 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fe1bb3-5d5d-4f81-a354-e5a94cfcbb61 admin admin] PATH_INFO: `/v3/projects/f8ff9813967748dfb7578d6215a8b8f7/users/2148db93191d4869b0777224588f3e29/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:09:09.397093 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fe1bb3-5d5d-4f81-a354-e5a94cfcbb61 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8ff9813967748dfb7578d6215a8b8f7, user_id=2148db93191d4869b0777224588f3e29, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:09:09.399717 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fe1bb3-5d5d-4f81-a354-e5a94cfcbb61 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:09:09.415543 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1152/2311] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 04:09:09 2022] PUT /identity/v3/projects/f8ff9813967748dfb7578d6215a8b8f7/users/2148db93191d4869b0777224588f3e29/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:09:09.419272 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd14f767-404f-4d03-9eef-035c165df71f admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:09:09.431338 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd14f767-404f-4d03-9eef-035c165df71f admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:09:09.509513 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc9d818-6bc9-4c5a-bdda-41c3bc0b06b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:09:09.510135 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc9d818-6bc9-4c5a-bdda-41c3bc0b06b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:09:09.510413 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc9d818-6bc9-4c5a-bdda-41c3bc0b06b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:09:09.510682 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc9d818-6bc9-4c5a-bdda-41c3bc0b06b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:09:09.511328 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc9d818-6bc9-4c5a-bdda-41c3bc0b06b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:09:09.512973 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc9d818-6bc9-4c5a-bdda-41c3bc0b06b7 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:09:09.538028 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1160/2312] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:09:09 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:09:09.539903 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fe1bb3-5d5d-4f81-a354-e5a94cfcbb61 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:09:09.550821 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fe1bb3-5d5d-4f81-a354-e5a94cfcbb61 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:09:09.555815 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5245b8d5-ba74-490e-9de0-174ddcdd25b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:09:09.556511 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5245b8d5-ba74-490e-9de0-174ddcdd25b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:09:09.556982 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5245b8d5-ba74-490e-9de0-174ddcdd25b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:09:09.557168 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5245b8d5-ba74-490e-9de0-174ddcdd25b3 admin admin] PATH_INFO: `/v3/projects/f8ff9813967748dfb7578d6215a8b8f7/users/2148db93191d4869b0777224588f3e29/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:09:09.559181 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5245b8d5-ba74-490e-9de0-174ddcdd25b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8ff9813967748dfb7578d6215a8b8f7, user_id=2148db93191d4869b0777224588f3e29, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:09:09.561859 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5245b8d5-ba74-490e-9de0-174ddcdd25b3 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:09:09.573019 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1153/2313] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 04:09:09 2022] PUT /identity/v3/projects/f8ff9813967748dfb7578d6215a8b8f7/users/2148db93191d4869b0777224588f3e29/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:09:14.196986 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85e3ac-3792-4570-af69-672d6fa7885e None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:09:14.197475 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85e3ac-3792-4570-af69-672d6fa7885e None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:09:14.197665 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e85e3ac-3792-4570-af69-672d6fa7885e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:09:14.236304 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-6e85e3ac-3792-4570-af69-672d6fa7885e None None] MFA Rules not processed for user `2148db93191d4869b0777224588f3e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:09:14.238457 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-6e85e3ac-3792-4570-af69-672d6fa7885e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:09:14.324122 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1161/2314] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:09:14 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:09:14.336974 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5245b8d5-ba74-490e-9de0-174ddcdd25b3 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:09:14.338758 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5245b8d5-ba74-490e-9de0-174ddcdd25b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:09:14.403861 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5245b8d5-ba74-490e-9de0-174ddcdd25b3 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:09:14.407508 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a881663-04a1-4814-bee4-0b99af330958 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50aab624ee9c4651a7cd42124bcc465f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:09:14.408150 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a881663-04a1-4814-bee4-0b99af330958 service glance] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:09:14.408977 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a881663-04a1-4814-bee4-0b99af330958 service glance] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:09:14.409246 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a881663-04a1-4814-bee4-0b99af330958 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:09:14.435343 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:09:14.469457 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a881663-04a1-4814-bee4-0b99af330958 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:09:14.471396 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a881663-04a1-4814-bee4-0b99af330958 service glance] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:09:14.480120 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1154/2315] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:09:14 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:09:14.611468 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe836463-23dd-4039-af31-0740ef269d38 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:09:14.611757 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe836463-23dd-4039-af31-0740ef269d38 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:09:14.612074 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe836463-23dd-4039-af31-0740ef269d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:09:14.619488 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:09:14.632651 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-fe836463-23dd-4039-af31-0740ef269d38 None None] MFA Rules not processed for user `da7ee607b66e43f3823d9d2faf373974`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:09:14.633854 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-fe836463-23dd-4039-af31-0740ef269d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:09:14.689676 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1162/2316] 10.222.0.26 () {58 vars in 1009 bytes} [Wed Nov 2 04:09:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:09:14.711094 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a881663-04a1-4814-bee4-0b99af330958 service glance] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:09:14.712539 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5a881663-04a1-4814-bee4-0b99af330958 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:09:14.780926 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a881663-04a1-4814-bee4-0b99af330958 service glance] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:09:14.786653 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e34648-afcf-4e73-9904-82e53221fdef service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5193c40a917b4105ab8b27aead1685f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:09:14.787264 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e34648-afcf-4e73-9904-82e53221fdef service swift] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:09:14.787544 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e34648-afcf-4e73-9904-82e53221fdef service swift] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:09:14.787819 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e34648-afcf-4e73-9904-82e53221fdef service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:09:14.795722 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e34648-afcf-4e73-9904-82e53221fdef service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:09:14.797425 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e34648-afcf-4e73-9904-82e53221fdef service swift] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:09:14.805460 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1155/2317] 10.222.0.26 () {60 vars in 1384 bytes} [Wed Nov 2 04:09:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 04:09:23.549173 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be821527-a956-4106-8a20-2a715ea2f127 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:09:23.549694 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be821527-a956-4106-8a20-2a715ea2f127 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:09:23.549913 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be821527-a956-4106-8a20-2a715ea2f127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:09:23.569315 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-be821527-a956-4106-8a20-2a715ea2f127 None None] MFA Rules not processed for user `8b56999a02524408b74371e6f8d158cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:09:23.570564 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-be821527-a956-4106-8a20-2a715ea2f127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:09:23.577307 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1163/2318] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:09:23 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 04:09:24.135363 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e34648-afcf-4e73-9904-82e53221fdef service swift] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:09:24.136980 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c6e34648-afcf-4e73-9904-82e53221fdef service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:09:24.208668 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e34648-afcf-4e73-9904-82e53221fdef service swift] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:09:24.213819 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc34bdd-18e0-4024-b0ac-4230575a18da service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4b9bd741574ddabadd4b7651ebc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:09:24.214437 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc34bdd-18e0-4024-b0ac-4230575a18da service placement] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:09:24.214720 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc34bdd-18e0-4024-b0ac-4230575a18da service placement] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:09:24.214996 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc34bdd-18e0-4024-b0ac-4230575a18da service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:09:24.216768 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7cc34bdd-18e0-4024-b0ac-4230575a18da service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:09:24.229525 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc34bdd-18e0-4024-b0ac-4230575a18da service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:09:24.231110 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc34bdd-18e0-4024-b0ac-4230575a18da service placement] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:09:24.241667 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1156/2319] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:09:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:09:25.777925 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a70cc-5fb7-4ec4-82d2-8b595b93a7a7 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:09:25.778412 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a70cc-5fb7-4ec4-82d2-8b595b93a7a7 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:09:25.778507 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a70cc-5fb7-4ec4-82d2-8b595b93a7a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:09:25.797765 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-662a70cc-5fb7-4ec4-82d2-8b595b93a7a7 None None] MFA Rules not processed for user `da7ee607b66e43f3823d9d2faf373974`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:09:25.798906 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-662a70cc-5fb7-4ec4-82d2-8b595b93a7a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:09:25.806676 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1164/2320] 10.222.0.26 () {58 vars in 1009 bytes} [Wed Nov 2 04:09:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:09:25.820882 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc34bdd-18e0-4024-b0ac-4230575a18da service placement] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:09:25.833841 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc34bdd-18e0-4024-b0ac-4230575a18da service placement] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:09:25.839127 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d1d51d-5949-4fc0-9364-c75960f99756 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5193c40a917b4105ab8b27aead1685f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:09:25.839800 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d1d51d-5949-4fc0-9364-c75960f99756 service swift] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:09:25.840178 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d1d51d-5949-4fc0-9364-c75960f99756 service swift] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:09:25.840523 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d1d51d-5949-4fc0-9364-c75960f99756 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:09:25.848887 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d1d51d-5949-4fc0-9364-c75960f99756 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:09:25.850527 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d1d51d-5949-4fc0-9364-c75960f99756 service swift] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:09:25.858799 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1157/2321] 10.222.0.26 () {60 vars in 1384 bytes} [Wed Nov 2 04:09:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 04:09:31.968860 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662a70cc-5fb7-4ec4-82d2-8b595b93a7a7 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:09:31.970273 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-662a70cc-5fb7-4ec4-82d2-8b595b93a7a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:09:32.038784 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662a70cc-5fb7-4ec4-82d2-8b595b93a7a7 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:09:32.042375 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7249ec9b-2d72-4108-9626-9784f4ad96b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:09:32.043017 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7249ec9b-2d72-4108-9626-9784f4ad96b5 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:09:32.043295 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7249ec9b-2d72-4108-9626-9784f4ad96b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:09:32.043567 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7249ec9b-2d72-4108-9626-9784f4ad96b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:09:32.045331 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-7249ec9b-2d72-4108-9626-9784f4ad96b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:09:32.121923 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7249ec9b-2d72-4108-9626-9784f4ad96b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:09:32.123646 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7249ec9b-2d72-4108-9626-9784f4ad96b5 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:09:32.136554 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1165/2322] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:09:31 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:09:35.375145 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d1d51d-5949-4fc0-9364-c75960f99756 service swift] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:09:35.377222 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-49d1d51d-5949-4fc0-9364-c75960f99756 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:09:35.425706 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:09:35.471454 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d1d51d-5949-4fc0-9364-c75960f99756 service swift] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:09:35.477203 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8f2c22-a8e0-47ab-b9b8-9820ca49a855 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:09:35.477824 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f2c22-a8e0-47ab-b9b8-9820ca49a855 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:09:35.478107 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f2c22-a8e0-47ab-b9b8-9820ca49a855 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:09:35.478453 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8f2c22-a8e0-47ab-b9b8-9820ca49a855 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:09:35.480132 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0c8f2c22-a8e0-47ab-b9b8-9820ca49a855 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:09:35.493515 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8f2c22-a8e0-47ab-b9b8-9820ca49a855 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:09:35.494694 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8f2c22-a8e0-47ab-b9b8-9820ca49a855 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:09:35.506641 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1158/2323] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:09:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:09:41.767723 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7249ec9b-2d72-4108-9626-9784f4ad96b5 service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:09:41.769243 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-7249ec9b-2d72-4108-9626-9784f4ad96b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:09:41.798242 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7249ec9b-2d72-4108-9626-9784f4ad96b5 service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:09:41.808967 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4972de41-8c9b-4340-b160-f4ddab8122ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:09:41.810054 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972de41-8c9b-4340-b160-f4ddab8122ac service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:09:41.810339 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972de41-8c9b-4340-b160-f4ddab8122ac service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:09:41.810613 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972de41-8c9b-4340-b160-f4ddab8122ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:09:41.812406 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-4972de41-8c9b-4340-b160-f4ddab8122ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:09:41.826271 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4972de41-8c9b-4340-b160-f4ddab8122ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:09:41.827722 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4972de41-8c9b-4340-b160-f4ddab8122ac service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:09:41.842150 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1166/2324] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:09:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:09:45.501200 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=20 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:09:51.836828 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:09:54.214700 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8f2c22-a8e0-47ab-b9b8-9820ca49a855 service nova] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:09:54.270366 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8f2c22-a8e0-47ab-b9b8-9820ca49a855 service nova] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:09:54.273976 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f4d94-f373-45f1-8410-ab0e8e81a7b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:09:54.274584 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f4d94-f373-45f1-8410-ab0e8e81a7b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:09:54.274859 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f4d94-f373-45f1-8410-ab0e8e81a7b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:09:54.275130 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59f4d94-f373-45f1-8410-ab0e8e81a7b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:09:54.276838 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a59f4d94-f373-45f1-8410-ab0e8e81a7b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:09:54.290331 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59f4d94-f373-45f1-8410-ab0e8e81a7b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:09:54.291732 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59f4d94-f373-45f1-8410-ab0e8e81a7b2 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:09:54.302672 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1159/2325] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:09:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:09:54.433060 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4972de41-8c9b-4340-b160-f4ddab8122ac service nova] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:09:54.442419 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4972de41-8c9b-4340-b160-f4ddab8122ac service nova] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:09:54.446013 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a5854d-9f55-41d7-b9c4-cdb00e8975ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:09:54.446612 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a5854d-9f55-41d7-b9c4-cdb00e8975ce service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:09:54.446895 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a5854d-9f55-41d7-b9c4-cdb00e8975ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:09:54.447168 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a5854d-9f55-41d7-b9c4-cdb00e8975ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:09:54.456659 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a5854d-9f55-41d7-b9c4-cdb00e8975ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:09:54.458624 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a5854d-9f55-41d7-b9c4-cdb00e8975ce service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:09:54.468341 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1167/2326] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:09:54 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:09:56.244744 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f4d94-f373-45f1-8410-ab0e8e81a7b2 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:09:56.259119 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59f4d94-f373-45f1-8410-ab0e8e81a7b2 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:09:56.266145 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4776e6-b7bc-4557-876d-7e62789b7691 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:09:56.266835 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4776e6-b7bc-4557-876d-7e62789b7691 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:09:56.267182 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4776e6-b7bc-4557-876d-7e62789b7691 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:09:56.267520 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4776e6-b7bc-4557-876d-7e62789b7691 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:09:56.269691 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-eb4776e6-b7bc-4557-876d-7e62789b7691 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:09:56.283990 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4776e6-b7bc-4557-876d-7e62789b7691 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:09:56.286081 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4776e6-b7bc-4557-876d-7e62789b7691 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:09:56.297794 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1160/2327] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:09:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:09:58.844418 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a5854d-9f55-41d7-b9c4-cdb00e8975ce service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:09:58.845846 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-67a5854d-9f55-41d7-b9c4-cdb00e8975ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:09:58.857018 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a5854d-9f55-41d7-b9c4-cdb00e8975ce service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:09:58.860668 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01188a90-3967-4d81-ba88-0a3dd29ba653 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:09:58.861258 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01188a90-3967-4d81-ba88-0a3dd29ba653 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:09:58.861532 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01188a90-3967-4d81-ba88-0a3dd29ba653 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:09:58.861796 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01188a90-3967-4d81-ba88-0a3dd29ba653 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:09:58.863969 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-01188a90-3967-4d81-ba88-0a3dd29ba653 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:09:58.873951 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01188a90-3967-4d81-ba88-0a3dd29ba653 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:09:58.875365 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01188a90-3967-4d81-ba88-0a3dd29ba653 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:09:58.893834 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1168/2328] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:09:58 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:10:00.997352 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4776e6-b7bc-4557-876d-7e62789b7691 service nova] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:10:01.007481 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4776e6-b7bc-4557-876d-7e62789b7691 service nova] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:10:01.011091 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cb0166-fcb3-462f-bbf2-f0c3631f2337 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:10:01.011688 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cb0166-fcb3-462f-bbf2-f0c3631f2337 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:10:01.011984 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cb0166-fcb3-462f-bbf2-f0c3631f2337 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:10:01.012261 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cb0166-fcb3-462f-bbf2-f0c3631f2337 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:10:01.013949 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-06cb0166-fcb3-462f-bbf2-f0c3631f2337 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:10:01.084891 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cb0166-fcb3-462f-bbf2-f0c3631f2337 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:10:01.086313 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cb0166-fcb3-462f-bbf2-f0c3631f2337 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:10:01.095282 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1161/2329] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:10:00 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:10:08.880331 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=12 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:10:11.091095 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=65 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:10:26.252689 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6d6ce8-1587-4a02-a6b5-1e86b569af47 None None] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:10:26.253257 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6d6ce8-1587-4a02-a6b5-1e86b569af47 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:10:26.253361 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6d6ce8-1587-4a02-a6b5-1e86b569af47 None None] PATH_INFO: `/` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:10:26.254889 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1169/2330] 10.222.0.26 () {54 vars in 921 bytes} [Wed Nov 2 04:10:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 04:10:26.261093 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4fa75a-817d-43f0-b02c-3a51c51c935c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:10:26.261872 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4fa75a-817d-43f0-b02c-3a51c51c935c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:10:26.262151 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4fa75a-817d-43f0-b02c-3a51c51c935c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:10:26.299904 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-0e4fa75a-817d-43f0-b02c-3a51c51c935c None None] MFA Rules not processed for user `f07e1b51b00f42b7b74c3689268635cb`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:10:26.301210 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0e4fa75a-817d-43f0-b02c-3a51c51c935c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:10:26.308673 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1162/2331] 10.222.0.26 () {58 vars in 1018 bytes} [Wed Nov 2 04:10:26 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 04:10:26.321370 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6d6ce8-1587-4a02-a6b5-1e86b569af47 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:10:26.332729 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6d6ce8-1587-4a02-a6b5-1e86b569af47 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:10:26.336267 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3affbd99-ab82-4a90-8ef4-e8ba520a2633 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:10:26.336860 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3affbd99-ab82-4a90-8ef4-e8ba520a2633 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:10:26.337133 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3affbd99-ab82-4a90-8ef4-e8ba520a2633 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:10:26.337398 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3affbd99-ab82-4a90-8ef4-e8ba520a2633 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:10:26.344910 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3affbd99-ab82-4a90-8ef4-e8ba520a2633 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:10:26.346351 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3affbd99-ab82-4a90-8ef4-e8ba520a2633 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:10:26.354761 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1170/2332] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:10:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:10:26.451390 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7748f6-5723-49d9-897e-79e04ca1e6cf None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:10:26.451677 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7748f6-5723-49d9-897e-79e04ca1e6cf None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:10:26.451973 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7748f6-5723-49d9-897e-79e04ca1e6cf None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:10:26.453360 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1163/2333] 10.222.0.26 () {54 vars in 921 bytes} [Wed Nov 2 04:10:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Nov 02 04:10:26.458932 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2957a916-70fd-499f-bc88-b5531d3b7310 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:10:26.459216 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2957a916-70fd-499f-bc88-b5531d3b7310 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:10:26.459497 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2957a916-70fd-499f-bc88-b5531d3b7310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:10:26.496848 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-2957a916-70fd-499f-bc88-b5531d3b7310 None None] MFA Rules not processed for user `44961b06151c4da382fcd54dd1f1e146`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:10:26.498234 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-2957a916-70fd-499f-bc88-b5531d3b7310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:10:26.507402 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1171/2334] 10.222.0.26 () {58 vars in 1018 bytes} [Wed Nov 2 04:10:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:10:26.518632 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7748f6-5723-49d9-897e-79e04ca1e6cf None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:10:26.530601 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7748f6-5723-49d9-897e-79e04ca1e6cf None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:10:26.534198 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e154f1fe-1d2b-4405-ba38-f9c99cb5d540 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:10:26.534797 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154f1fe-1d2b-4405-ba38-f9c99cb5d540 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:10:26.535075 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154f1fe-1d2b-4405-ba38-f9c99cb5d540 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:10:26.535349 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154f1fe-1d2b-4405-ba38-f9c99cb5d540 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:10:26.546049 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e154f1fe-1d2b-4405-ba38-f9c99cb5d540 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:10:26.547461 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e154f1fe-1d2b-4405-ba38-f9c99cb5d540 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:10:26.559118 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1164/2335] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:10:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:10:36.494516 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=15 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:10:36.553572 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=15 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:11:01.356793 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2957a916-70fd-499f-bc88-b5531d3b7310 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:01.367289 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2957a916-70fd-499f-bc88-b5531d3b7310 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:01.370946 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37187ad4-d5a5-41ad-b1e5-a0d4abffcab1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:01.371545 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37187ad4-d5a5-41ad-b1e5-a0d4abffcab1 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:01.371820 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37187ad4-d5a5-41ad-b1e5-a0d4abffcab1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:01.372195 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37187ad4-d5a5-41ad-b1e5-a0d4abffcab1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:01.374159 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-37187ad4-d5a5-41ad-b1e5-a0d4abffcab1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:11:01.435809 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37187ad4-d5a5-41ad-b1e5-a0d4abffcab1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:01.437321 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37187ad4-d5a5-41ad-b1e5-a0d4abffcab1 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:01.447857 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1172/2336] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:11:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:11:04.527037 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e154f1fe-1d2b-4405-ba38-f9c99cb5d540 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:04.528686 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e154f1fe-1d2b-4405-ba38-f9c99cb5d540 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:11:04.557368 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e154f1fe-1d2b-4405-ba38-f9c99cb5d540 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:04.563226 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8efd394-5e71-432b-ab68-da645eb1a30b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4b9bd741574ddabadd4b7651ebc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:04.564295 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8efd394-5e71-432b-ab68-da645eb1a30b service placement] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:04.564736 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8efd394-5e71-432b-ab68-da645eb1a30b service placement] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:04.565181 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8efd394-5e71-432b-ab68-da645eb1a30b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:04.566979 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f8efd394-5e71-432b-ab68-da645eb1a30b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:11:04.587173 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8efd394-5e71-432b-ab68-da645eb1a30b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:04.589597 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8efd394-5e71-432b-ab68-da645eb1a30b service placement] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:04.601731 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37187ad4-d5a5-41ad-b1e5-a0d4abffcab1 service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:04.603273 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-37187ad4-d5a5-41ad-b1e5-a0d4abffcab1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:11:04.604153 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1165/2337] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:11:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:11:04.617451 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37187ad4-d5a5-41ad-b1e5-a0d4abffcab1 service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:04.625241 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8f2853-3317-416b-b81d-1b2868df2163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:04.625990 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f2853-3317-416b-b81d-1b2868df2163 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:04.626340 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f2853-3317-416b-b81d-1b2868df2163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:04.626626 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8f2853-3317-416b-b81d-1b2868df2163 admin admin] PATH_INFO: `/v3/users/18355f4225c9427dbb38fde84115d7b2` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:04.627943 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8f2853-3317-416b-b81d-1b2868df2163 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18355f4225c9427dbb38fde84115d7b2)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:04.630245 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8f2853-3317-416b-b81d-1b2868df2163 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:04.678151 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-cf8f2853-3317-416b-b81d-1b2868df2163 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18355f4225c9427dbb38fde84115d7b2'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:04.678810 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-cf8f2853-3317-416b-b81d-1b2868df2163 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18355f4225c9427dbb38fde84115d7b2'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:04.679427 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-cf8f2853-3317-416b-b81d-1b2868df2163 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18355f4225c9427dbb38fde84115d7b2'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:04.694574 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-cf8f2853-3317-416b-b81d-1b2868df2163 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18355f4225c9427dbb38fde84115d7b2'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:04.701937 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-cf8f2853-3317-416b-b81d-1b2868df2163 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18355f4225c9427dbb38fde84115d7b2'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:04.714528 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1173/2338] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 04:11:04 2022] DELETE /identity/v3/users/18355f4225c9427dbb38fde84115d7b2 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 04:11:04.890212 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8efd394-5e71-432b-ab68-da645eb1a30b service placement] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:04.901834 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8efd394-5e71-432b-ab68-da645eb1a30b service placement] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:04.957676 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24013a5-f47d-4c38-b3ac-3b045111eac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:04.958303 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24013a5-f47d-4c38-b3ac-3b045111eac4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:04.958577 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24013a5-f47d-4c38-b3ac-3b045111eac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:04.958847 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24013a5-f47d-4c38-b3ac-3b045111eac4 admin admin] PATH_INFO: `/v3/projects/85403204e27c40ffb0155dd8aba3d77b` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:04.960520 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24013a5-f47d-4c38-b3ac-3b045111eac4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85403204e27c40ffb0155dd8aba3d77b)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:04.962433 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24013a5-f47d-4c38-b3ac-3b045111eac4 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:05.007374 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a24013a5-f47d-4c38-b3ac-3b045111eac4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85403204e27c40ffb0155dd8aba3d77b', 'request_id': 'req-a24013a5-f47d-4c38-b3ac-3b045111eac4'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:05.013659 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a24013a5-f47d-4c38-b3ac-3b045111eac4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85403204e27c40ffb0155dd8aba3d77b', 'request_id': 'req-a24013a5-f47d-4c38-b3ac-3b045111eac4'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:05.020935 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a24013a5-f47d-4c38-b3ac-3b045111eac4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85403204e27c40ffb0155dd8aba3d77b', 'request_id': 'req-a24013a5-f47d-4c38-b3ac-3b045111eac4'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:05.032049 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a24013a5-f47d-4c38-b3ac-3b045111eac4 admin admin] The token cache is being invalidate because project 85403204e27c40ffb0155dd8aba3d77b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 04:11:05.032337 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a24013a5-f47d-4c38-b3ac-3b045111eac4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:05.032983 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a24013a5-f47d-4c38-b3ac-3b045111eac4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:05.034475 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1166/2339] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 04:11:04 2022] DELETE /identity/v3/projects/85403204e27c40ffb0155dd8aba3d77b => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:11:05.040343 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3670a-c94f-4b2a-b08d-58bd8ec4f046 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:05.040650 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3670a-c94f-4b2a-b08d-58bd8ec4f046 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:05.040918 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c3670a-c94f-4b2a-b08d-58bd8ec4f046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:05.060462 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-26c3670a-c94f-4b2a-b08d-58bd8ec4f046 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:11:05.061661 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-26c3670a-c94f-4b2a-b08d-58bd8ec4f046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:11:05.118851 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1174/2340] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:11:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 04:11:05.126695 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be399a-6afe-4bc8-9a8b-8ac54723010a None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:05.127011 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be399a-6afe-4bc8-9a8b-8ac54723010a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:05.127311 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9be399a-6afe-4bc8-9a8b-8ac54723010a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:05.149342 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d9be399a-6afe-4bc8-9a8b-8ac54723010a None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:11:05.150457 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d9be399a-6afe-4bc8-9a8b-8ac54723010a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:11:05.158391 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1167/2341] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:11:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:11:05.162460 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c3670a-c94f-4b2a-b08d-58bd8ec4f046 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:05.172090 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c3670a-c94f-4b2a-b08d-58bd8ec4f046 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:05.176621 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb875ab-fc3d-4653-be3f-cb9b9c654c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:05.177239 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb875ab-fc3d-4653-be3f-cb9b9c654c64 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:05.177514 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb875ab-fc3d-4653-be3f-cb9b9c654c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:05.177783 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb875ab-fc3d-4653-be3f-cb9b9c654c64 admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:05.178414 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb875ab-fc3d-4653-be3f-cb9b9c654c64 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 04:11:05.178712 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb875ab-fc3d-4653-be3f-cb9b9c654c64 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:05.180422 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb875ab-fc3d-4653-be3f-cb9b9c654c64 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:05.181835 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:11:05.195181 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1175/2342] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 04:11:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 04:11:05.199311 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9be399a-6afe-4bc8-9a8b-8ac54723010a None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:05.208468 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9be399a-6afe-4bc8-9a8b-8ac54723010a None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:05.213683 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029e42a8-ac8d-4b1f-b196-1882c2d7c2f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:05.214269 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e42a8-ac8d-4b1f-b196-1882c2d7c2f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:05.214545 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e42a8-ac8d-4b1f-b196-1882c2d7c2f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:05.214816 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029e42a8-ac8d-4b1f-b196-1882c2d7c2f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:05.215439 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029e42a8-ac8d-4b1f-b196-1882c2d7c2f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:05.217338 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029e42a8-ac8d-4b1f-b196-1882c2d7c2f3 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:05.243069 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1168/2343] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 04:11:05 2022] POST /identity/v3/projects => generated 371 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:11:05.247104 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb875ab-fc3d-4653-be3f-cb9b9c654c64 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:05.257639 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb875ab-fc3d-4653-be3f-cb9b9c654c64 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:05.311671 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e459e011-5dce-4044-a65b-c33793b1555c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:05.312295 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e459e011-5dce-4044-a65b-c33793b1555c admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:05.312574 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e459e011-5dce-4044-a65b-c33793b1555c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:05.312840 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e459e011-5dce-4044-a65b-c33793b1555c admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:05.313470 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e459e011-5dce-4044-a65b-c33793b1555c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:05.315363 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e459e011-5dce-4044-a65b-c33793b1555c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:05.341601 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1176/2344] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 04:11:05 2022] POST /identity/v3/users => generated 286 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 04:11:05.342738 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029e42a8-ac8d-4b1f-b196-1882c2d7c2f3 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:05.352912 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029e42a8-ac8d-4b1f-b196-1882c2d7c2f3 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:05.357498 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cce589-0897-4826-b61d-b10c3a4a4339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:05.358102 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cce589-0897-4826-b61d-b10c3a4a4339 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:05.358379 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cce589-0897-4826-b61d-b10c3a4a4339 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:05.358701 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cce589-0897-4826-b61d-b10c3a4a4339 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:05.359281 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cce589-0897-4826-b61d-b10c3a4a4339 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:05.360900 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cce589-0897-4826-b61d-b10c3a4a4339 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:05.372044 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1169/2345] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:11:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:11:05.376313 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e459e011-5dce-4044-a65b-c33793b1555c admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:05.387472 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e459e011-5dce-4044-a65b-c33793b1555c admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:05.392431 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03969e93-0301-4924-8cee-619705875094 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:05.392905 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03969e93-0301-4924-8cee-619705875094 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:05.393188 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03969e93-0301-4924-8cee-619705875094 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:05.393463 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03969e93-0301-4924-8cee-619705875094 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:05.394112 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03969e93-0301-4924-8cee-619705875094 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:05.395739 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03969e93-0301-4924-8cee-619705875094 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:05.406863 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1177/2346] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:11:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:11:05.410866 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cce589-0897-4826-b61d-b10c3a4a4339 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:05.421005 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cce589-0897-4826-b61d-b10c3a4a4339 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:05.425791 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8acc90-d701-4ed5-8f5a-f4504b798dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:05.426544 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8acc90-d701-4ed5-8f5a-f4504b798dcd admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:05.426872 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8acc90-d701-4ed5-8f5a-f4504b798dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:05.427152 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8acc90-d701-4ed5-8f5a-f4504b798dcd admin admin] PATH_INFO: `/v3/projects/cb05a034bfbd4df2ba29ede9d20f3ab0/users/44eff86e53bd480485bb1cd37d3db96f/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:05.443968 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8acc90-d701-4ed5-8f5a-f4504b798dcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb05a034bfbd4df2ba29ede9d20f3ab0, user_id=44eff86e53bd480485bb1cd37d3db96f, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:05.446609 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8acc90-d701-4ed5-8f5a-f4504b798dcd admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:05.460658 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1170/2347] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 04:11:05 2022] PUT /identity/v3/projects/cb05a034bfbd4df2ba29ede9d20f3ab0/users/44eff86e53bd480485bb1cd37d3db96f/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:11:05.470800 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549909c-556f-4e38-b103-3ef16a45ebbd None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:05.471085 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549909c-556f-4e38-b103-3ef16a45ebbd None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:05.471357 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8549909c-556f-4e38-b103-3ef16a45ebbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:05.499440 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-8549909c-556f-4e38-b103-3ef16a45ebbd None None] MFA Rules not processed for user `44eff86e53bd480485bb1cd37d3db96f`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:11:05.500689 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-8549909c-556f-4e38-b103-3ef16a45ebbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:11:05.566095 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1178/2348] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:11:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:11:05.579876 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8acc90-d701-4ed5-8f5a-f4504b798dcd admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:05.582144 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ce8acc90-d701-4ed5-8f5a-f4504b798dcd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:11:05.596889 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:11:05.664423 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8acc90-d701-4ed5-8f5a-f4504b798dcd admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:05.670187 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f4b1b4-ffc7-4435-bea0-3d99d00ee223 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:05.670858 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4b1b4-ffc7-4435-bea0-3d99d00ee223 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:05.671135 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4b1b4-ffc7-4435-bea0-3d99d00ee223 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:05.671405 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f4b1b4-ffc7-4435-bea0-3d99d00ee223 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:05.691646 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f4b1b4-ffc7-4435-bea0-3d99d00ee223 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:05.693267 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f4b1b4-ffc7-4435-bea0-3d99d00ee223 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:05.703108 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1171/2349] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:11:05 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:11:08.471618 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8549909c-556f-4e38-b103-3ef16a45ebbd None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:08.543370 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8549909c-556f-4e38-b103-3ef16a45ebbd None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:08.548519 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c168a6bf-fcb0-4130-8174-58da6d22c4e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:08.549337 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c168a6bf-fcb0-4130-8174-58da6d22c4e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:08.549689 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c168a6bf-fcb0-4130-8174-58da6d22c4e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:08.550033 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c168a6bf-fcb0-4130-8174-58da6d22c4e2 admin admin] PATH_INFO: `/v3/users/44eff86e53bd480485bb1cd37d3db96f` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:08.551196 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c168a6bf-fcb0-4130-8174-58da6d22c4e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44eff86e53bd480485bb1cd37d3db96f)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:08.553187 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c168a6bf-fcb0-4130-8174-58da6d22c4e2 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:08.555699 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:11:08.604148 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-c168a6bf-fcb0-4130-8174-58da6d22c4e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44eff86e53bd480485bb1cd37d3db96f'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:08.604807 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-c168a6bf-fcb0-4130-8174-58da6d22c4e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44eff86e53bd480485bb1cd37d3db96f'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:08.605417 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-c168a6bf-fcb0-4130-8174-58da6d22c4e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44eff86e53bd480485bb1cd37d3db96f'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:08.617366 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-c168a6bf-fcb0-4130-8174-58da6d22c4e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44eff86e53bd480485bb1cd37d3db96f'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:08.626381 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-c168a6bf-fcb0-4130-8174-58da6d22c4e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44eff86e53bd480485bb1cd37d3db96f'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:08.639165 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1179/2350] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 04:11:08 2022] DELETE /identity/v3/users/44eff86e53bd480485bb1cd37d3db96f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:11:08.643387 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b9b778-8923-449d-9e00-5495c7dc794c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:08.643880 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b9b778-8923-449d-9e00-5495c7dc794c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:08.644191 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b9b778-8923-449d-9e00-5495c7dc794c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:08.664276 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-94b9b778-8923-449d-9e00-5495c7dc794c None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:11:08.666977 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-94b9b778-8923-449d-9e00-5495c7dc794c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:11:08.725551 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1172/2351] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:11:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:11:08.734768 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c168a6bf-fcb0-4130-8174-58da6d22c4e2 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:08.736226 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-c168a6bf-fcb0-4130-8174-58da6d22c4e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:11:08.795434 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c168a6bf-fcb0-4130-8174-58da6d22c4e2 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:08.799066 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3529003-fae0-43db-a182-e1539831e3dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:08.799671 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3529003-fae0-43db-a182-e1539831e3dd service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:08.799970 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3529003-fae0-43db-a182-e1539831e3dd service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:08.800240 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3529003-fae0-43db-a182-e1539831e3dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:08.809666 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3529003-fae0-43db-a182-e1539831e3dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:08.811614 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3529003-fae0-43db-a182-e1539831e3dd service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:08.822493 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1180/2352] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:11:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:11:09.109300 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b9b778-8923-449d-9e00-5495c7dc794c None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:09.110901 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-94b9b778-8923-449d-9e00-5495c7dc794c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:11:09.129107 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b9b778-8923-449d-9e00-5495c7dc794c None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:09.135840 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750361da-e890-4ff4-ae7a-65fe3c816afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:09.137965 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750361da-e890-4ff4-ae7a-65fe3c816afd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:09.138322 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750361da-e890-4ff4-ae7a-65fe3c816afd admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:09.138730 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750361da-e890-4ff4-ae7a-65fe3c816afd admin admin] PATH_INFO: `/v3/projects/cb05a034bfbd4df2ba29ede9d20f3ab0` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:09.140306 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750361da-e890-4ff4-ae7a-65fe3c816afd admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb05a034bfbd4df2ba29ede9d20f3ab0)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:09.143463 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750361da-e890-4ff4-ae7a-65fe3c816afd admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:09.200314 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-750361da-e890-4ff4-ae7a-65fe3c816afd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb05a034bfbd4df2ba29ede9d20f3ab0', 'request_id': 'req-750361da-e890-4ff4-ae7a-65fe3c816afd'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:09.208597 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-750361da-e890-4ff4-ae7a-65fe3c816afd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb05a034bfbd4df2ba29ede9d20f3ab0', 'request_id': 'req-750361da-e890-4ff4-ae7a-65fe3c816afd'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:09.217489 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-750361da-e890-4ff4-ae7a-65fe3c816afd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb05a034bfbd4df2ba29ede9d20f3ab0', 'request_id': 'req-750361da-e890-4ff4-ae7a-65fe3c816afd'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:09.229997 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-750361da-e890-4ff4-ae7a-65fe3c816afd admin admin] The token cache is being invalidate because project cb05a034bfbd4df2ba29ede9d20f3ab0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 04:11:09.230375 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-750361da-e890-4ff4-ae7a-65fe3c816afd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:09.231262 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-750361da-e890-4ff4-ae7a-65fe3c816afd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:09.233240 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1173/2353] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 04:11:09 2022] DELETE /identity/v3/projects/cb05a034bfbd4df2ba29ede9d20f3ab0 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:11:09.239513 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac2633-50f7-411e-83bf-26c2783f5b83 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:09.239878 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac2633-50f7-411e-83bf-26c2783f5b83 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:09.241009 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ac2633-50f7-411e-83bf-26c2783f5b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:09.263847 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-70ac2633-50f7-411e-83bf-26c2783f5b83 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:11:09.265097 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-70ac2633-50f7-411e-83bf-26c2783f5b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:11:09.362186 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1181/2354] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:11:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:11:09.372195 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe6174-c168-4747-a214-22abd4d38884 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:09.372487 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe6174-c168-4747-a214-22abd4d38884 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:09.373018 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe6174-c168-4747-a214-22abd4d38884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:09.394456 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-0efe6174-c168-4747-a214-22abd4d38884 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:11:09.395656 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0efe6174-c168-4747-a214-22abd4d38884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:11:09.408305 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1174/2355] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:11:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:11:09.409713 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ac2633-50f7-411e-83bf-26c2783f5b83 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:09.419987 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ac2633-50f7-411e-83bf-26c2783f5b83 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:09.426248 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436e7e33-6f9d-4cc1-873f-1083ab6fe017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:09.426973 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e7e33-6f9d-4cc1-873f-1083ab6fe017 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:09.427346 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e7e33-6f9d-4cc1-873f-1083ab6fe017 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:09.428341 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e7e33-6f9d-4cc1-873f-1083ab6fe017 admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:09.429113 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436e7e33-6f9d-4cc1-873f-1083ab6fe017 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 04:11:09.429418 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436e7e33-6f9d-4cc1-873f-1083ab6fe017 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:09.431443 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436e7e33-6f9d-4cc1-873f-1083ab6fe017 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:09.444784 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1182/2356] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 04:11:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 04:11:09.452895 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efe6174-c168-4747-a214-22abd4d38884 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:09.466503 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efe6174-c168-4747-a214-22abd4d38884 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:09.473786 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7a8841-ee4b-47cf-8e06-0163da06966c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:09.475056 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a8841-ee4b-47cf-8e06-0163da06966c admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:09.475424 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a8841-ee4b-47cf-8e06-0163da06966c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:09.475705 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7a8841-ee4b-47cf-8e06-0163da06966c admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:09.476440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7a8841-ee4b-47cf-8e06-0163da06966c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:09.478409 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7a8841-ee4b-47cf-8e06-0163da06966c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:09.496489 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:11:09.513660 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1175/2357] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 04:11:09 2022] POST /identity/v3/projects => generated 387 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:11:09.517781 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436e7e33-6f9d-4cc1-873f-1083ab6fe017 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:09.529451 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436e7e33-6f9d-4cc1-873f-1083ab6fe017 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:09.538715 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:11:09.595422 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7379f28a-1ab9-4b14-b7e7-9520058e3edf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:09.596732 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7379f28a-1ab9-4b14-b7e7-9520058e3edf admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:09.597156 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7379f28a-1ab9-4b14-b7e7-9520058e3edf admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:09.597438 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7379f28a-1ab9-4b14-b7e7-9520058e3edf admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:09.598650 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7379f28a-1ab9-4b14-b7e7-9520058e3edf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:09.601446 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7379f28a-1ab9-4b14-b7e7-9520058e3edf admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:09.632156 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1183/2358] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 04:11:09 2022] POST /identity/v3/users => generated 294 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:11:09.638358 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7a8841-ee4b-47cf-8e06-0163da06966c admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:09.650403 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7a8841-ee4b-47cf-8e06-0163da06966c admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:09.655435 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c817558c-7c7a-4a7d-949f-e642e627b649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:09.655933 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c817558c-7c7a-4a7d-949f-e642e627b649 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:09.656244 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c817558c-7c7a-4a7d-949f-e642e627b649 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:09.656519 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c817558c-7c7a-4a7d-949f-e642e627b649 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:09.657220 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c817558c-7c7a-4a7d-949f-e642e627b649 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:09.659078 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c817558c-7c7a-4a7d-949f-e642e627b649 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:09.673870 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1176/2359] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:11:09 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:11:09.675170 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7379f28a-1ab9-4b14-b7e7-9520058e3edf admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:09.687356 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7379f28a-1ab9-4b14-b7e7-9520058e3edf admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:09.692853 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd49db0-e5c7-45a9-818e-9f6ad878e293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:09.695190 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd49db0-e5c7-45a9-818e-9f6ad878e293 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:09.695483 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd49db0-e5c7-45a9-818e-9f6ad878e293 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:09.695733 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd49db0-e5c7-45a9-818e-9f6ad878e293 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:09.696634 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd49db0-e5c7-45a9-818e-9f6ad878e293 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:09.698247 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd49db0-e5c7-45a9-818e-9f6ad878e293 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:09.712403 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1184/2360] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:11:09 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 04:11:09.716618 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c817558c-7c7a-4a7d-949f-e642e627b649 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:09.729886 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c817558c-7c7a-4a7d-949f-e642e627b649 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:09.735514 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45920999-5678-4085-bbe4-927c3c0d74b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:09.736360 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45920999-5678-4085-bbe4-927c3c0d74b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:09.736709 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45920999-5678-4085-bbe4-927c3c0d74b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:09.737055 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45920999-5678-4085-bbe4-927c3c0d74b2 admin admin] PATH_INFO: `/v3/projects/51e3dc8f4dc14630baa2fba67324e987/users/85eaa57ce4f44a3e99fdf5a65f41e7eb/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:09.761542 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45920999-5678-4085-bbe4-927c3c0d74b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51e3dc8f4dc14630baa2fba67324e987, user_id=85eaa57ce4f44a3e99fdf5a65f41e7eb, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:09.765395 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45920999-5678-4085-bbe4-927c3c0d74b2 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:09.780432 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1177/2361] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 04:11:09 2022] PUT /identity/v3/projects/51e3dc8f4dc14630baa2fba67324e987/users/85eaa57ce4f44a3e99fdf5a65f41e7eb/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:11:09.791634 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09693e2a-82be-4d70-977d-fdd819ec6c7a None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:09.792047 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09693e2a-82be-4d70-977d-fdd819ec6c7a None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:09.792414 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09693e2a-82be-4d70-977d-fdd819ec6c7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:09.823363 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-09693e2a-82be-4d70-977d-fdd819ec6c7a None None] MFA Rules not processed for user `85eaa57ce4f44a3e99fdf5a65f41e7eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:11:09.824744 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-09693e2a-82be-4d70-977d-fdd819ec6c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:11:09.898101 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1185/2362] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:11:09 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:11:09.902414 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45920999-5678-4085-bbe4-927c3c0d74b2 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:09.973880 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45920999-5678-4085-bbe4-927c3c0d74b2 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:09.979818 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f73a4e-52ee-4d35-b433-60476545c81c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:09.980364 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f73a4e-52ee-4d35-b433-60476545c81c admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:09.980646 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f73a4e-52ee-4d35-b433-60476545c81c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:09.980919 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f73a4e-52ee-4d35-b433-60476545c81c admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:09.981573 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f73a4e-52ee-4d35-b433-60476545c81c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:09.983495 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f73a4e-52ee-4d35-b433-60476545c81c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:10.012315 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1178/2363] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 04:11:09 2022] POST /identity/v3/projects => generated 387 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 04:11:10.016889 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09693e2a-82be-4d70-977d-fdd819ec6c7a None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:10.028496 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09693e2a-82be-4d70-977d-fdd819ec6c7a None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:10.086296 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0eebd0-0b6b-4968-8b37-ad2810e52517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:10.087012 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0eebd0-0b6b-4968-8b37-ad2810e52517 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:10.087294 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0eebd0-0b6b-4968-8b37-ad2810e52517 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:10.087637 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0eebd0-0b6b-4968-8b37-ad2810e52517 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:10.088379 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0eebd0-0b6b-4968-8b37-ad2810e52517 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:10.090448 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0eebd0-0b6b-4968-8b37-ad2810e52517 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:10.110696 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1186/2364] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 04:11:10 2022] POST /identity/v3/users => generated 300 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 04:11:10.114840 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f73a4e-52ee-4d35-b433-60476545c81c admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:10.125745 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f73a4e-52ee-4d35-b433-60476545c81c admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:10.130524 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983a2fbe-94f6-47ba-ae0c-36d35c81d6a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:10.131227 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983a2fbe-94f6-47ba-ae0c-36d35c81d6a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:10.131510 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983a2fbe-94f6-47ba-ae0c-36d35c81d6a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:10.131845 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983a2fbe-94f6-47ba-ae0c-36d35c81d6a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:10.132600 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983a2fbe-94f6-47ba-ae0c-36d35c81d6a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:10.134353 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983a2fbe-94f6-47ba-ae0c-36d35c81d6a2 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:10.145654 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1179/2365] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:11:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:11:10.150290 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0eebd0-0b6b-4968-8b37-ad2810e52517 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:10.160156 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0eebd0-0b6b-4968-8b37-ad2810e52517 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:10.165338 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9603d5-e0d6-4713-a11d-98d0cddfcfba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:10.165983 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9603d5-e0d6-4713-a11d-98d0cddfcfba admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:10.166263 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9603d5-e0d6-4713-a11d-98d0cddfcfba admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:10.166536 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9603d5-e0d6-4713-a11d-98d0cddfcfba admin admin] PATH_INFO: `/v3/projects/9717a25f7198499983f853f18443080e/users/d22209c8c16e4d98bbc1457f0558b79e/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:10.169546 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:11:10.182117 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9603d5-e0d6-4713-a11d-98d0cddfcfba admin admin] RBAC: Authorizing `identity:create_grant(project_id=9717a25f7198499983f853f18443080e, user_id=d22209c8c16e4d98bbc1457f0558b79e, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:10.184851 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9603d5-e0d6-4713-a11d-98d0cddfcfba admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:10.197873 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1187/2366] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 04:11:10 2022] PUT /identity/v3/projects/9717a25f7198499983f853f18443080e/users/d22209c8c16e4d98bbc1457f0558b79e/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:11:10.202003 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983a2fbe-94f6-47ba-ae0c-36d35c81d6a2 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:10.212419 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983a2fbe-94f6-47ba-ae0c-36d35c81d6a2 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:10.269436 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f38ab0-eff8-4d38-abf5-0b3625bac137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:10.269970 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f38ab0-eff8-4d38-abf5-0b3625bac137 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:10.270251 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f38ab0-eff8-4d38-abf5-0b3625bac137 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:10.270522 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f38ab0-eff8-4d38-abf5-0b3625bac137 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:10.271153 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f38ab0-eff8-4d38-abf5-0b3625bac137 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:10.273416 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f38ab0-eff8-4d38-abf5-0b3625bac137 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:10.282984 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1180/2367] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:11:10 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:11:10.287189 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9603d5-e0d6-4713-a11d-98d0cddfcfba admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:10.297297 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9603d5-e0d6-4713-a11d-98d0cddfcfba admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:10.301892 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8032eb83-6072-4e5f-a2a0-6e7db81da983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:10.302532 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8032eb83-6072-4e5f-a2a0-6e7db81da983 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:10.302814 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8032eb83-6072-4e5f-a2a0-6e7db81da983 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:10.303087 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8032eb83-6072-4e5f-a2a0-6e7db81da983 admin admin] PATH_INFO: `/v3/projects/9717a25f7198499983f853f18443080e/users/d22209c8c16e4d98bbc1457f0558b79e/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:10.305060 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8032eb83-6072-4e5f-a2a0-6e7db81da983 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9717a25f7198499983f853f18443080e, user_id=d22209c8c16e4d98bbc1457f0558b79e, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:10.307684 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8032eb83-6072-4e5f-a2a0-6e7db81da983 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:10.317845 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1188/2368] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 04:11:10 2022] PUT /identity/v3/projects/9717a25f7198499983f853f18443080e/users/d22209c8c16e4d98bbc1457f0558b79e/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:11:10.329591 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a92ab51-ddc2-41d8-b993-988503e7adc7 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:10.329879 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a92ab51-ddc2-41d8-b993-988503e7adc7 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:10.330178 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a92ab51-ddc2-41d8-b993-988503e7adc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:10.358494 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-0a92ab51-ddc2-41d8-b993-988503e7adc7 None None] MFA Rules not processed for user `d22209c8c16e4d98bbc1457f0558b79e`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:11:10.359803 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0a92ab51-ddc2-41d8-b993-988503e7adc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:11:10.376304 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:11:10.428718 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1181/2369] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:11:10 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:11:10.439870 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8032eb83-6072-4e5f-a2a0-6e7db81da983 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:10.441591 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-8032eb83-6072-4e5f-a2a0-6e7db81da983 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:11:10.499190 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8032eb83-6072-4e5f-a2a0-6e7db81da983 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:10.502885 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6ab713-d89e-46a8-99d1-971f6a3f2287 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:10.503484 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6ab713-d89e-46a8-99d1-971f6a3f2287 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:10.503759 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6ab713-d89e-46a8-99d1-971f6a3f2287 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:10.504088 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6ab713-d89e-46a8-99d1-971f6a3f2287 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:10.513401 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6ab713-d89e-46a8-99d1-971f6a3f2287 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:10.514818 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6ab713-d89e-46a8-99d1-971f6a3f2287 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:10.524753 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1189/2370] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:11:10 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:11:10.553314 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a92ab51-ddc2-41d8-b993-988503e7adc7 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:10.614767 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a92ab51-ddc2-41d8-b993-988503e7adc7 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:10.620197 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5d8751-aabf-4237-a46d-79012f57b158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:10.620819 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d8751-aabf-4237-a46d-79012f57b158 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:10.621095 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d8751-aabf-4237-a46d-79012f57b158 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:10.621365 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d8751-aabf-4237-a46d-79012f57b158 admin admin] PATH_INFO: `/v3/users/85eaa57ce4f44a3e99fdf5a65f41e7eb` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:10.622480 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5d8751-aabf-4237-a46d-79012f57b158 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85eaa57ce4f44a3e99fdf5a65f41e7eb)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:10.625039 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5d8751-aabf-4237-a46d-79012f57b158 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:10.669449 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7b5d8751-aabf-4237-a46d-79012f57b158 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85eaa57ce4f44a3e99fdf5a65f41e7eb'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:10.676350 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7b5d8751-aabf-4237-a46d-79012f57b158 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85eaa57ce4f44a3e99fdf5a65f41e7eb'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:10.677031 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7b5d8751-aabf-4237-a46d-79012f57b158 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85eaa57ce4f44a3e99fdf5a65f41e7eb'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:10.689271 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7b5d8751-aabf-4237-a46d-79012f57b158 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85eaa57ce4f44a3e99fdf5a65f41e7eb'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:10.697961 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7b5d8751-aabf-4237-a46d-79012f57b158 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85eaa57ce4f44a3e99fdf5a65f41e7eb'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:10.703851 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1182/2371] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 04:11:10 2022] DELETE /identity/v3/users/85eaa57ce4f44a3e99fdf5a65f41e7eb => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 04:11:10.708327 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08042379-bd3e-4f67-98d5-f1bcc4e62887 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:10.708614 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08042379-bd3e-4f67-98d5-f1bcc4e62887 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:10.708937 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08042379-bd3e-4f67-98d5-f1bcc4e62887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:10.727420 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-08042379-bd3e-4f67-98d5-f1bcc4e62887 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:11:10.728628 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-08042379-bd3e-4f67-98d5-f1bcc4e62887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:11:10.789100 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1190/2372] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:11:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 04:11:10.798525 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5d8751-aabf-4237-a46d-79012f57b158 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:10.800403 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7b5d8751-aabf-4237-a46d-79012f57b158 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:11:10.836758 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:11:10.861026 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5d8751-aabf-4237-a46d-79012f57b158 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:10.865701 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf66354-0bae-452f-8939-d779428877c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:10.866323 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf66354-0bae-452f-8939-d779428877c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:10.866596 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf66354-0bae-452f-8939-d779428877c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:10.866869 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf66354-0bae-452f-8939-d779428877c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:10.876477 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf66354-0bae-452f-8939-d779428877c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:10.877966 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf66354-0bae-452f-8939-d779428877c0 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:10.890625 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1183/2373] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:11:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:11:11.169126 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08042379-bd3e-4f67-98d5-f1bcc4e62887 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:11.170539 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-08042379-bd3e-4f67-98d5-f1bcc4e62887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:11:11.182714 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08042379-bd3e-4f67-98d5-f1bcc4e62887 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:11.187488 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcc52ae-4673-4bc6-a7b9-53e6d23ff8bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:11.188158 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc52ae-4673-4bc6-a7b9-53e6d23ff8bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:11.188435 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc52ae-4673-4bc6-a7b9-53e6d23ff8bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:11.188703 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcc52ae-4673-4bc6-a7b9-53e6d23ff8bb admin admin] PATH_INFO: `/v3/projects/51e3dc8f4dc14630baa2fba67324e987` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:11.189807 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcc52ae-4673-4bc6-a7b9-53e6d23ff8bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=51e3dc8f4dc14630baa2fba67324e987)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:11.191710 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcc52ae-4673-4bc6-a7b9-53e6d23ff8bb admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:11.238272 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-afcc52ae-4673-4bc6-a7b9-53e6d23ff8bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51e3dc8f4dc14630baa2fba67324e987', 'request_id': 'req-afcc52ae-4673-4bc6-a7b9-53e6d23ff8bb'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:11.244708 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-afcc52ae-4673-4bc6-a7b9-53e6d23ff8bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51e3dc8f4dc14630baa2fba67324e987', 'request_id': 'req-afcc52ae-4673-4bc6-a7b9-53e6d23ff8bb'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:11.250930 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-afcc52ae-4673-4bc6-a7b9-53e6d23ff8bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51e3dc8f4dc14630baa2fba67324e987', 'request_id': 'req-afcc52ae-4673-4bc6-a7b9-53e6d23ff8bb'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:11.259498 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:11:11.264075 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-afcc52ae-4673-4bc6-a7b9-53e6d23ff8bb admin admin] The token cache is being invalidate because project 51e3dc8f4dc14630baa2fba67324e987 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 04:11:11.264467 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-afcc52ae-4673-4bc6-a7b9-53e6d23ff8bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:11.265918 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-afcc52ae-4673-4bc6-a7b9-53e6d23ff8bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:11.267846 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1191/2374] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 04:11:11 2022] DELETE /identity/v3/projects/51e3dc8f4dc14630baa2fba67324e987 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:11:11.272127 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf66354-0bae-452f-8939-d779428877c0 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:11.281562 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf66354-0bae-452f-8939-d779428877c0 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:11.338678 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cd88f7-6a83-4587-b6d9-4386804ac75d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:11.339305 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd88f7-6a83-4587-b6d9-4386804ac75d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:11.339584 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd88f7-6a83-4587-b6d9-4386804ac75d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:11.339856 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cd88f7-6a83-4587-b6d9-4386804ac75d admin admin] PATH_INFO: `/v3/users/d22209c8c16e4d98bbc1457f0558b79e` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:11.341003 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cd88f7-6a83-4587-b6d9-4386804ac75d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d22209c8c16e4d98bbc1457f0558b79e)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:11.343295 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cd88f7-6a83-4587-b6d9-4386804ac75d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:11.388590 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b3cd88f7-6a83-4587-b6d9-4386804ac75d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd22209c8c16e4d98bbc1457f0558b79e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:11.395496 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b3cd88f7-6a83-4587-b6d9-4386804ac75d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd22209c8c16e4d98bbc1457f0558b79e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:11.396179 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b3cd88f7-6a83-4587-b6d9-4386804ac75d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd22209c8c16e4d98bbc1457f0558b79e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:11.408409 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b3cd88f7-6a83-4587-b6d9-4386804ac75d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd22209c8c16e4d98bbc1457f0558b79e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:11.416359 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b3cd88f7-6a83-4587-b6d9-4386804ac75d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd22209c8c16e4d98bbc1457f0558b79e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:11.422363 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1184/2375] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 04:11:11 2022] DELETE /identity/v3/users/d22209c8c16e4d98bbc1457f0558b79e => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:11:11.690834 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcc52ae-4673-4bc6-a7b9-53e6d23ff8bb admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:11.692339 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-afcc52ae-4673-4bc6-a7b9-53e6d23ff8bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:11:11.705291 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcc52ae-4673-4bc6-a7b9-53e6d23ff8bb admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:11.763401 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de86b30-c329-4437-9fe9-282f8acf4686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:11.764074 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de86b30-c329-4437-9fe9-282f8acf4686 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:11.764354 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de86b30-c329-4437-9fe9-282f8acf4686 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:11.764626 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de86b30-c329-4437-9fe9-282f8acf4686 admin admin] PATH_INFO: `/v3/projects/9717a25f7198499983f853f18443080e` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:11.765746 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de86b30-c329-4437-9fe9-282f8acf4686 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9717a25f7198499983f853f18443080e)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:11.767675 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de86b30-c329-4437-9fe9-282f8acf4686 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:11.813173 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2de86b30-c329-4437-9fe9-282f8acf4686 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9717a25f7198499983f853f18443080e', 'request_id': 'req-2de86b30-c329-4437-9fe9-282f8acf4686'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:11.819108 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2de86b30-c329-4437-9fe9-282f8acf4686 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9717a25f7198499983f853f18443080e', 'request_id': 'req-2de86b30-c329-4437-9fe9-282f8acf4686'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:11.825116 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2de86b30-c329-4437-9fe9-282f8acf4686 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9717a25f7198499983f853f18443080e', 'request_id': 'req-2de86b30-c329-4437-9fe9-282f8acf4686'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:11.837571 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2de86b30-c329-4437-9fe9-282f8acf4686 admin admin] The token cache is being invalidate because project 9717a25f7198499983f853f18443080e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 04:11:11.837863 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2de86b30-c329-4437-9fe9-282f8acf4686 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:11.838482 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2de86b30-c329-4437-9fe9-282f8acf4686 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:11.840007 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1192/2376] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 04:11:11 2022] DELETE /identity/v3/projects/9717a25f7198499983f853f18443080e => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:11:11.844863 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d75a0-3773-488a-9357-b59c06ce1e4d None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:11.845326 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d75a0-3773-488a-9357-b59c06ce1e4d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:11.845417 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d75a0-3773-488a-9357-b59c06ce1e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:11.864586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-f92d75a0-3773-488a-9357-b59c06ce1e4d None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:11:11.865837 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f92d75a0-3773-488a-9357-b59c06ce1e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:11:11.913099 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:11:11.925565 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1185/2377] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:11:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:11:11.934332 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad6694b-2079-4f18-bf4d-a65a9b261f33 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:11.934621 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad6694b-2079-4f18-bf4d-a65a9b261f33 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:11.934899 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad6694b-2079-4f18-bf4d-a65a9b261f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:11.955052 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-dad6694b-2079-4f18-bf4d-a65a9b261f33 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:11:11.956280 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-dad6694b-2079-4f18-bf4d-a65a9b261f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:11:11.964792 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1193/2378] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:11:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:11:11.968867 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92d75a0-3773-488a-9357-b59c06ce1e4d None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:11.978693 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92d75a0-3773-488a-9357-b59c06ce1e4d None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:11.983233 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24f788e-1fcb-44a9-87d1-70e7242b4048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:11.983830 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f788e-1fcb-44a9-87d1-70e7242b4048 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:11.984153 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f788e-1fcb-44a9-87d1-70e7242b4048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:11.984428 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24f788e-1fcb-44a9-87d1-70e7242b4048 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:11.985067 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24f788e-1fcb-44a9-87d1-70e7242b4048 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 04:11:11.985371 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24f788e-1fcb-44a9-87d1-70e7242b4048 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:11.986957 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24f788e-1fcb-44a9-87d1-70e7242b4048 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:11.998720 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1186/2379] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 04:11:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 04:11:12.002771 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad6694b-2079-4f18-bf4d-a65a9b261f33 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:12.012706 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad6694b-2079-4f18-bf4d-a65a9b261f33 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:12.017488 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa9447d-e39c-40c0-95d9-f73ae5a2576d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:12.018089 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa9447d-e39c-40c0-95d9-f73ae5a2576d admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:12.018399 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa9447d-e39c-40c0-95d9-f73ae5a2576d admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:12.018672 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa9447d-e39c-40c0-95d9-f73ae5a2576d admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:12.019311 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa9447d-e39c-40c0-95d9-f73ae5a2576d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:12.021398 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa9447d-e39c-40c0-95d9-f73ae5a2576d admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:12.049941 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1194/2380] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 04:11:12 2022] POST /identity/v3/projects => generated 407 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:11:12.054282 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24f788e-1fcb-44a9-87d1-70e7242b4048 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:12.067353 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24f788e-1fcb-44a9-87d1-70e7242b4048 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:12.125556 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02b9394-3b4e-4e63-b317-318f910dbce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:12.126095 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02b9394-3b4e-4e63-b317-318f910dbce0 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:12.126374 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02b9394-3b4e-4e63-b317-318f910dbce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:12.126646 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02b9394-3b4e-4e63-b317-318f910dbce0 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:12.127277 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02b9394-3b4e-4e63-b317-318f910dbce0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:12.129210 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02b9394-3b4e-4e63-b317-318f910dbce0 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:12.153898 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1187/2381] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 04:11:12 2022] POST /identity/v3/users => generated 304 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 04:11:12.156177 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa9447d-e39c-40c0-95d9-f73ae5a2576d admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:12.166373 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa9447d-e39c-40c0-95d9-f73ae5a2576d admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:12.170980 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd99aa74-4a41-4680-91f0-fed97cc51f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:12.171580 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd99aa74-4a41-4680-91f0-fed97cc51f2d admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:12.171858 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd99aa74-4a41-4680-91f0-fed97cc51f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:12.172174 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd99aa74-4a41-4680-91f0-fed97cc51f2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:12.172800 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd99aa74-4a41-4680-91f0-fed97cc51f2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:12.174391 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd99aa74-4a41-4680-91f0-fed97cc51f2d admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:12.184877 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1195/2382] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:11:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:11:12.188907 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02b9394-3b4e-4e63-b317-318f910dbce0 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:12.199660 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02b9394-3b4e-4e63-b317-318f910dbce0 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:12.204275 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8687605-ae39-481e-ac1b-b89acdb2d7ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:12.204875 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8687605-ae39-481e-ac1b-b89acdb2d7ce admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:12.205149 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8687605-ae39-481e-ac1b-b89acdb2d7ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:12.205421 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8687605-ae39-481e-ac1b-b89acdb2d7ce admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:12.206379 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8687605-ae39-481e-ac1b-b89acdb2d7ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:12.207974 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8687605-ae39-481e-ac1b-b89acdb2d7ce admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:12.218957 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1188/2383] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:11:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 04:11:12.223264 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd99aa74-4a41-4680-91f0-fed97cc51f2d admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:12.233111 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd99aa74-4a41-4680-91f0-fed97cc51f2d admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:12.238141 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f93edad-83ac-4606-bfd3-16bb8847b530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:12.238853 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f93edad-83ac-4606-bfd3-16bb8847b530 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:12.239120 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f93edad-83ac-4606-bfd3-16bb8847b530 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:12.239390 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f93edad-83ac-4606-bfd3-16bb8847b530 admin admin] PATH_INFO: `/v3/projects/4fdb7f529ea543aca2b7cce6e0c47edd/users/5189b657b07e4d6ea9eb1fbf044c4396/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:12.255250 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f93edad-83ac-4606-bfd3-16bb8847b530 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fdb7f529ea543aca2b7cce6e0c47edd, user_id=5189b657b07e4d6ea9eb1fbf044c4396, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:12.262760 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f93edad-83ac-4606-bfd3-16bb8847b530 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:12.275648 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1196/2384] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 04:11:12 2022] PUT /identity/v3/projects/4fdb7f529ea543aca2b7cce6e0c47edd/users/5189b657b07e4d6ea9eb1fbf044c4396/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:11:12.285680 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093fa0d-0fe1-4eb2-b501-f7783d0f40b8 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:12.285970 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093fa0d-0fe1-4eb2-b501-f7783d0f40b8 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:12.286244 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3093fa0d-0fe1-4eb2-b501-f7783d0f40b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:12.315002 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-3093fa0d-0fe1-4eb2-b501-f7783d0f40b8 None None] MFA Rules not processed for user `5189b657b07e4d6ea9eb1fbf044c4396`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:11:12.316259 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3093fa0d-0fe1-4eb2-b501-f7783d0f40b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:11:12.379279 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1189/2385] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:11:12 2022] POST /identity/v3/auth/tokens => generated 3284 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:11:12.384304 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f93edad-83ac-4606-bfd3-16bb8847b530 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:12.410820 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:11:12.447069 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f93edad-83ac-4606-bfd3-16bb8847b530 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:12.451679 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b920b97f-e554-446f-8146-61b4b82a91ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:12.452313 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b920b97f-e554-446f-8146-61b4b82a91ce admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:12.452591 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b920b97f-e554-446f-8146-61b4b82a91ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:12.452862 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b920b97f-e554-446f-8146-61b4b82a91ce admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:12.453491 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b920b97f-e554-446f-8146-61b4b82a91ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:12.455375 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b920b97f-e554-446f-8146-61b4b82a91ce admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:12.480751 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1197/2386] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 04:11:12 2022] POST /identity/v3/projects => generated 407 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:11:12.484808 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3093fa0d-0fe1-4eb2-b501-f7783d0f40b8 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:12.495999 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3093fa0d-0fe1-4eb2-b501-f7783d0f40b8 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:12.544651 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:11:12.555259 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7dd614-5fd2-4d4b-8a24-d50736f2d266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:12.555874 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7dd614-5fd2-4d4b-8a24-d50736f2d266 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:12.556181 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7dd614-5fd2-4d4b-8a24-d50736f2d266 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:12.556453 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7dd614-5fd2-4d4b-8a24-d50736f2d266 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:12.557093 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7dd614-5fd2-4d4b-8a24-d50736f2d266 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:12.558989 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7dd614-5fd2-4d4b-8a24-d50736f2d266 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:12.579384 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1190/2387] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 04:11:12 2022] POST /identity/v3/users => generated 310 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:11:12.583423 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b920b97f-e554-446f-8146-61b4b82a91ce admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:12.594211 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b920b97f-e554-446f-8146-61b4b82a91ce admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:12.598764 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b55df5-1af4-4712-8432-b31e5f051661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:12.599360 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b55df5-1af4-4712-8432-b31e5f051661 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:12.599640 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b55df5-1af4-4712-8432-b31e5f051661 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:12.599915 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b55df5-1af4-4712-8432-b31e5f051661 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:12.600592 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b55df5-1af4-4712-8432-b31e5f051661 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:12.602168 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b55df5-1af4-4712-8432-b31e5f051661 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:12.612531 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1198/2388] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:11:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:11:12.616512 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7dd614-5fd2-4d4b-8a24-d50736f2d266 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:12.626753 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7dd614-5fd2-4d4b-8a24-d50736f2d266 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:12.631984 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb8c1c0-71d9-4a81-8cda-cee43124918e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:12.632570 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb8c1c0-71d9-4a81-8cda-cee43124918e admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:12.632847 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb8c1c0-71d9-4a81-8cda-cee43124918e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:12.633137 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb8c1c0-71d9-4a81-8cda-cee43124918e admin admin] PATH_INFO: `/v3/projects/48d692e270a441f08950c581e2a0d9d8/users/0cb5f3f8049d486f8829082f65a2e785/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:12.648331 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb8c1c0-71d9-4a81-8cda-cee43124918e admin admin] RBAC: Authorizing `identity:create_grant(project_id=48d692e270a441f08950c581e2a0d9d8, user_id=0cb5f3f8049d486f8829082f65a2e785, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:12.650943 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb8c1c0-71d9-4a81-8cda-cee43124918e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:12.663217 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1191/2389] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 04:11:12 2022] PUT /identity/v3/projects/48d692e270a441f08950c581e2a0d9d8/users/0cb5f3f8049d486f8829082f65a2e785/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 04:11:12.667082 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b55df5-1af4-4712-8432-b31e5f051661 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:12.676849 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b55df5-1af4-4712-8432-b31e5f051661 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:12.731748 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542510f9-2784-43f0-97fc-0d0b18a12b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:12.732439 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542510f9-2784-43f0-97fc-0d0b18a12b49 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:12.732718 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542510f9-2784-43f0-97fc-0d0b18a12b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:12.733091 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542510f9-2784-43f0-97fc-0d0b18a12b49 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:12.733972 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542510f9-2784-43f0-97fc-0d0b18a12b49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:12.735311 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542510f9-2784-43f0-97fc-0d0b18a12b49 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:12.747425 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1199/2390] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:11:12 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:11:12.749595 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb8c1c0-71d9-4a81-8cda-cee43124918e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:12.760259 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb8c1c0-71d9-4a81-8cda-cee43124918e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:12.764781 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724cda22-dbd8-4b4e-ac37-36eb388ca279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:12.765416 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724cda22-dbd8-4b4e-ac37-36eb388ca279 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:12.765694 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724cda22-dbd8-4b4e-ac37-36eb388ca279 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:12.765967 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724cda22-dbd8-4b4e-ac37-36eb388ca279 admin admin] PATH_INFO: `/v3/projects/48d692e270a441f08950c581e2a0d9d8/users/0cb5f3f8049d486f8829082f65a2e785/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:12.767857 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724cda22-dbd8-4b4e-ac37-36eb388ca279 admin admin] RBAC: Authorizing `identity:create_grant(project_id=48d692e270a441f08950c581e2a0d9d8, user_id=0cb5f3f8049d486f8829082f65a2e785, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:12.770492 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724cda22-dbd8-4b4e-ac37-36eb388ca279 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:12.781030 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1192/2391] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 04:11:12 2022] PUT /identity/v3/projects/48d692e270a441f08950c581e2a0d9d8/users/0cb5f3f8049d486f8829082f65a2e785/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:11:12.790900 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9479d2f4-bcaf-4513-952e-dc93761666c2 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:12.791183 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9479d2f4-bcaf-4513-952e-dc93761666c2 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:12.791458 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9479d2f4-bcaf-4513-952e-dc93761666c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:12.818862 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-9479d2f4-bcaf-4513-952e-dc93761666c2 None None] MFA Rules not processed for user `0cb5f3f8049d486f8829082f65a2e785`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:11:12.820363 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-9479d2f4-bcaf-4513-952e-dc93761666c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:11:12.890629 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1200/2392] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:11:12 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:11:12.900944 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724cda22-dbd8-4b4e-ac37-36eb388ca279 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:12.902686 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-724cda22-dbd8-4b4e-ac37-36eb388ca279 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:11:12.960314 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724cda22-dbd8-4b4e-ac37-36eb388ca279 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:12.963989 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c16befe-58ec-4a8b-a724-ef0fd0bfbd47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:12.964593 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c16befe-58ec-4a8b-a724-ef0fd0bfbd47 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:12.964874 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c16befe-58ec-4a8b-a724-ef0fd0bfbd47 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:12.965149 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c16befe-58ec-4a8b-a724-ef0fd0bfbd47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:12.974076 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c16befe-58ec-4a8b-a724-ef0fd0bfbd47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:12.975510 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c16befe-58ec-4a8b-a724-ef0fd0bfbd47 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:12.985716 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1193/2393] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:11:12 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:11:13.149903 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9479d2f4-bcaf-4513-952e-dc93761666c2 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:13.160468 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9479d2f4-bcaf-4513-952e-dc93761666c2 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:13.164310 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa68dc1-67ac-4477-b797-6bc8cc65855b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:13.165062 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa68dc1-67ac-4477-b797-6bc8cc65855b service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:13.165454 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa68dc1-67ac-4477-b797-6bc8cc65855b service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:13.165798 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa68dc1-67ac-4477-b797-6bc8cc65855b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:13.183348 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:11:13.226721 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa68dc1-67ac-4477-b797-6bc8cc65855b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:13.228193 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa68dc1-67ac-4477-b797-6bc8cc65855b service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:13.237199 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1201/2394] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:11:13 2022] GET /identity/v3/auth/tokens => generated 3284 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:11:14.574626 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99d70be-200b-489e-8ae6-42f3cf613226 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:14.575077 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99d70be-200b-489e-8ae6-42f3cf613226 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:14.575196 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99d70be-200b-489e-8ae6-42f3cf613226 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:14.594311 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-b99d70be-200b-489e-8ae6-42f3cf613226 None None] MFA Rules not processed for user `da7ee607b66e43f3823d9d2faf373974`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:11:14.595523 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b99d70be-200b-489e-8ae6-42f3cf613226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:11:14.658260 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1194/2395] 10.222.0.26 () {58 vars in 967 bytes} [Wed Nov 2 04:11:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:11:14.701049 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa68dc1-67ac-4477-b797-6bc8cc65855b service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:14.702465 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-6fa68dc1-67ac-4477-b797-6bc8cc65855b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:11:14.775319 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa68dc1-67ac-4477-b797-6bc8cc65855b service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:14.780584 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8722cc4-b3d3-43c7-a928-3e786be31cb4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5193c40a917b4105ab8b27aead1685f8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:14.781198 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8722cc4-b3d3-43c7-a928-3e786be31cb4 service swift] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:14.781479 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8722cc4-b3d3-43c7-a928-3e786be31cb4 service swift] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:14.781748 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8722cc4-b3d3-43c7-a928-3e786be31cb4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:14.793077 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8722cc4-b3d3-43c7-a928-3e786be31cb4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:14.795056 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8722cc4-b3d3-43c7-a928-3e786be31cb4 service swift] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:14.805103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1202/2396] 10.222.0.26 () {60 vars in 1384 bytes} [Wed Nov 2 04:11:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Nov 02 04:11:21.521376 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99d70be-200b-489e-8ae6-42f3cf613226 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:21.525881 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b99d70be-200b-489e-8ae6-42f3cf613226 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:11:21.591557 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:11:21.613647 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99d70be-200b-489e-8ae6-42f3cf613226 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:21.619068 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49664cdf-c5e9-417f-8522-407e81888023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:21.619717 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49664cdf-c5e9-417f-8522-407e81888023 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:21.620028 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49664cdf-c5e9-417f-8522-407e81888023 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:21.620305 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49664cdf-c5e9-417f-8522-407e81888023 admin admin] PATH_INFO: `/v3/users/8b56999a02524408b74371e6f8d158cf` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:21.621838 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49664cdf-c5e9-417f-8522-407e81888023 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b56999a02524408b74371e6f8d158cf)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:21.624460 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49664cdf-c5e9-417f-8522-407e81888023 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:21.677273 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-49664cdf-c5e9-417f-8522-407e81888023 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b56999a02524408b74371e6f8d158cf'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:21.684125 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-49664cdf-c5e9-417f-8522-407e81888023 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b56999a02524408b74371e6f8d158cf'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:21.684795 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-49664cdf-c5e9-417f-8522-407e81888023 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b56999a02524408b74371e6f8d158cf'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:21.696436 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-49664cdf-c5e9-417f-8522-407e81888023 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b56999a02524408b74371e6f8d158cf'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:21.703919 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-49664cdf-c5e9-417f-8522-407e81888023 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b56999a02524408b74371e6f8d158cf'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:21.710442 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1195/2397] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 04:11:21 2022] DELETE /identity/v3/users/8b56999a02524408b74371e6f8d158cf => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:11:21.882269 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8722cc4-b3d3-43c7-a928-3e786be31cb4 service swift] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:21.897513 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8722cc4-b3d3-43c7-a928-3e786be31cb4 service swift] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:21.957717 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a10b9b-3fd4-4215-8eba-1ec75af91061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:21.958290 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a10b9b-3fd4-4215-8eba-1ec75af91061 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:21.958563 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a10b9b-3fd4-4215-8eba-1ec75af91061 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:21.958830 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a10b9b-3fd4-4215-8eba-1ec75af91061 admin admin] PATH_INFO: `/v3/projects/561a9720212b46e2bb93367218e0f112` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:21.960590 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a10b9b-3fd4-4215-8eba-1ec75af91061 admin admin] RBAC: Authorizing `identity:delete_project(project_id=561a9720212b46e2bb93367218e0f112)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:21.962511 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a10b9b-3fd4-4215-8eba-1ec75af91061 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:22.011000 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-41a10b9b-3fd4-4215-8eba-1ec75af91061 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '561a9720212b46e2bb93367218e0f112', 'request_id': 'req-41a10b9b-3fd4-4215-8eba-1ec75af91061'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:22.012184 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:11:22.018029 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-41a10b9b-3fd4-4215-8eba-1ec75af91061 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '561a9720212b46e2bb93367218e0f112', 'request_id': 'req-41a10b9b-3fd4-4215-8eba-1ec75af91061'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:22.024966 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-41a10b9b-3fd4-4215-8eba-1ec75af91061 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '561a9720212b46e2bb93367218e0f112', 'request_id': 'req-41a10b9b-3fd4-4215-8eba-1ec75af91061'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:22.038508 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-41a10b9b-3fd4-4215-8eba-1ec75af91061 admin admin] The token cache is being invalidate because project 561a9720212b46e2bb93367218e0f112 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 04:11:22.038802 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-41a10b9b-3fd4-4215-8eba-1ec75af91061 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:22.039427 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-41a10b9b-3fd4-4215-8eba-1ec75af91061 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:22.041009 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1203/2398] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 04:11:21 2022] DELETE /identity/v3/projects/561a9720212b46e2bb93367218e0f112 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 04:11:22.045938 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49664cdf-c5e9-417f-8522-407e81888023 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:22.047615 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-49664cdf-c5e9-417f-8522-407e81888023 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:11:22.060888 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49664cdf-c5e9-417f-8522-407e81888023 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:22.116177 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a98161-5fe1-4e16-a5ee-843c92ed6241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:22.116801 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a98161-5fe1-4e16-a5ee-843c92ed6241 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:22.117080 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a98161-5fe1-4e16-a5ee-843c92ed6241 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:22.117350 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a98161-5fe1-4e16-a5ee-843c92ed6241 admin admin] PATH_INFO: `/v3/users/2148db93191d4869b0777224588f3e29` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:22.118456 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a98161-5fe1-4e16-a5ee-843c92ed6241 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2148db93191d4869b0777224588f3e29)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:22.120422 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a98161-5fe1-4e16-a5ee-843c92ed6241 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:22.167861 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-37a98161-5fe1-4e16-a5ee-843c92ed6241 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2148db93191d4869b0777224588f3e29'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:22.174664 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-37a98161-5fe1-4e16-a5ee-843c92ed6241 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2148db93191d4869b0777224588f3e29'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:22.175310 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-37a98161-5fe1-4e16-a5ee-843c92ed6241 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2148db93191d4869b0777224588f3e29'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:22.186185 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:11:22.187405 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-37a98161-5fe1-4e16-a5ee-843c92ed6241 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2148db93191d4869b0777224588f3e29'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:22.195841 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-37a98161-5fe1-4e16-a5ee-843c92ed6241 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2148db93191d4869b0777224588f3e29'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:22.202528 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1196/2399] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 04:11:22 2022] DELETE /identity/v3/users/2148db93191d4869b0777224588f3e29 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:11:22.374975 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a10b9b-3fd4-4215-8eba-1ec75af91061 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:22.376433 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-41a10b9b-3fd4-4215-8eba-1ec75af91061 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:11:22.391714 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a10b9b-3fd4-4215-8eba-1ec75af91061 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:22.448571 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f120539-2864-4367-b806-da11e8368341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:22.449204 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f120539-2864-4367-b806-da11e8368341 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:22.449484 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f120539-2864-4367-b806-da11e8368341 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:22.449757 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f120539-2864-4367-b806-da11e8368341 admin admin] PATH_INFO: `/v3/projects/f8ff9813967748dfb7578d6215a8b8f7` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:22.450874 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f120539-2864-4367-b806-da11e8368341 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8ff9813967748dfb7578d6215a8b8f7)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:22.452819 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f120539-2864-4367-b806-da11e8368341 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:22.498632 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-7f120539-2864-4367-b806-da11e8368341 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8ff9813967748dfb7578d6215a8b8f7', 'request_id': 'req-7f120539-2864-4367-b806-da11e8368341'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:22.504195 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-7f120539-2864-4367-b806-da11e8368341 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8ff9813967748dfb7578d6215a8b8f7', 'request_id': 'req-7f120539-2864-4367-b806-da11e8368341'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:22.511175 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-7f120539-2864-4367-b806-da11e8368341 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8ff9813967748dfb7578d6215a8b8f7', 'request_id': 'req-7f120539-2864-4367-b806-da11e8368341'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:22.523242 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-7f120539-2864-4367-b806-da11e8368341 admin admin] The token cache is being invalidate because project f8ff9813967748dfb7578d6215a8b8f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 04:11:22.523535 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-7f120539-2864-4367-b806-da11e8368341 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:22.524226 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-7f120539-2864-4367-b806-da11e8368341 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:22.525731 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1204/2400] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 04:11:22 2022] DELETE /identity/v3/projects/f8ff9813967748dfb7578d6215a8b8f7 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:11:22.848634 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a98161-5fe1-4e16-a5ee-843c92ed6241 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:22.850579 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-37a98161-5fe1-4e16-a5ee-843c92ed6241 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:11:22.863484 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a98161-5fe1-4e16-a5ee-843c92ed6241 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:22.920349 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974f32b9-c3a4-431b-91b5-ed91d4ca1cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:22.920980 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974f32b9-c3a4-431b-91b5-ed91d4ca1cb3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:22.921263 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974f32b9-c3a4-431b-91b5-ed91d4ca1cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:22.921536 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974f32b9-c3a4-431b-91b5-ed91d4ca1cb3 admin admin] PATH_INFO: `/v3/users/5189b657b07e4d6ea9eb1fbf044c4396` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:22.922648 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974f32b9-c3a4-431b-91b5-ed91d4ca1cb3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5189b657b07e4d6ea9eb1fbf044c4396)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:22.924603 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974f32b9-c3a4-431b-91b5-ed91d4ca1cb3 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:22.970806 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-974f32b9-c3a4-431b-91b5-ed91d4ca1cb3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5189b657b07e4d6ea9eb1fbf044c4396'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:22.978340 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-974f32b9-c3a4-431b-91b5-ed91d4ca1cb3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5189b657b07e4d6ea9eb1fbf044c4396'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:22.979301 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-974f32b9-c3a4-431b-91b5-ed91d4ca1cb3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5189b657b07e4d6ea9eb1fbf044c4396'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:22.991815 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-974f32b9-c3a4-431b-91b5-ed91d4ca1cb3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5189b657b07e4d6ea9eb1fbf044c4396'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:22.999123 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-974f32b9-c3a4-431b-91b5-ed91d4ca1cb3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5189b657b07e4d6ea9eb1fbf044c4396'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:23.005097 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1197/2401] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 04:11:22 2022] DELETE /identity/v3/users/5189b657b07e4d6ea9eb1fbf044c4396 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:11:23.010581 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63898329-c9f5-440a-91df-7f9517355a38 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:23.011075 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63898329-c9f5-440a-91df-7f9517355a38 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:23.011177 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63898329-c9f5-440a-91df-7f9517355a38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:23.030463 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-63898329-c9f5-440a-91df-7f9517355a38 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:11:23.031664 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-63898329-c9f5-440a-91df-7f9517355a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:11:23.089965 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1205/2402] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:11:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:11:23.099642 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974f32b9-c3a4-431b-91b5-ed91d4ca1cb3 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:23.101725 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-974f32b9-c3a4-431b-91b5-ed91d4ca1cb3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:11:23.157381 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974f32b9-c3a4-431b-91b5-ed91d4ca1cb3 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:23.161015 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9815f9-c718-4690-a017-b9f2f11dda24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:23.161616 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9815f9-c718-4690-a017-b9f2f11dda24 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:23.161893 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9815f9-c718-4690-a017-b9f2f11dda24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:23.162163 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9815f9-c718-4690-a017-b9f2f11dda24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:23.171577 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9815f9-c718-4690-a017-b9f2f11dda24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:23.173029 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9815f9-c718-4690-a017-b9f2f11dda24 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:23.182803 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1198/2403] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:11:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:11:23.452757 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63898329-c9f5-440a-91df-7f9517355a38 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:23.454248 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-63898329-c9f5-440a-91df-7f9517355a38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:11:23.467202 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63898329-c9f5-440a-91df-7f9517355a38 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:23.472107 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d9ef7f-adbd-4202-966b-13091a07763b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:23.472757 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d9ef7f-adbd-4202-966b-13091a07763b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:23.473035 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d9ef7f-adbd-4202-966b-13091a07763b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:23.473310 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d9ef7f-adbd-4202-966b-13091a07763b admin admin] PATH_INFO: `/v3/projects/4fdb7f529ea543aca2b7cce6e0c47edd` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:23.474563 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d9ef7f-adbd-4202-966b-13091a07763b admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fdb7f529ea543aca2b7cce6e0c47edd)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:23.477025 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d9ef7f-adbd-4202-966b-13091a07763b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:23.479877 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:11:23.525400 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-32d9ef7f-adbd-4202-966b-13091a07763b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fdb7f529ea543aca2b7cce6e0c47edd', 'request_id': 'req-32d9ef7f-adbd-4202-966b-13091a07763b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:23.532682 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-32d9ef7f-adbd-4202-966b-13091a07763b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fdb7f529ea543aca2b7cce6e0c47edd', 'request_id': 'req-32d9ef7f-adbd-4202-966b-13091a07763b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:23.539623 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-32d9ef7f-adbd-4202-966b-13091a07763b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fdb7f529ea543aca2b7cce6e0c47edd', 'request_id': 'req-32d9ef7f-adbd-4202-966b-13091a07763b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:23.553747 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-32d9ef7f-adbd-4202-966b-13091a07763b admin admin] The token cache is being invalidate because project 4fdb7f529ea543aca2b7cce6e0c47edd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 04:11:23.554036 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-32d9ef7f-adbd-4202-966b-13091a07763b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:23.554655 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-32d9ef7f-adbd-4202-966b-13091a07763b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:23.556198 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1206/2404] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 04:11:23 2022] DELETE /identity/v3/projects/4fdb7f529ea543aca2b7cce6e0c47edd => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:11:23.560120 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9815f9-c718-4690-a017-b9f2f11dda24 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:23.569655 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9815f9-c718-4690-a017-b9f2f11dda24 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:23.577274 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:11:23.627325 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2f820e-97cc-4fc1-a40a-391b1b8cd447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:23.627969 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2f820e-97cc-4fc1-a40a-391b1b8cd447 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:23.628248 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2f820e-97cc-4fc1-a40a-391b1b8cd447 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:23.628516 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2f820e-97cc-4fc1-a40a-391b1b8cd447 admin admin] PATH_INFO: `/v3/users/0cb5f3f8049d486f8829082f65a2e785` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:23.629666 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2f820e-97cc-4fc1-a40a-391b1b8cd447 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0cb5f3f8049d486f8829082f65a2e785)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:23.631539 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2f820e-97cc-4fc1-a40a-391b1b8cd447 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:23.675703 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ae2f820e-97cc-4fc1-a40a-391b1b8cd447 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0cb5f3f8049d486f8829082f65a2e785'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:23.683189 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ae2f820e-97cc-4fc1-a40a-391b1b8cd447 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0cb5f3f8049d486f8829082f65a2e785'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:23.683852 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ae2f820e-97cc-4fc1-a40a-391b1b8cd447 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0cb5f3f8049d486f8829082f65a2e785'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:23.695592 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ae2f820e-97cc-4fc1-a40a-391b1b8cd447 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0cb5f3f8049d486f8829082f65a2e785'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:23.703154 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ae2f820e-97cc-4fc1-a40a-391b1b8cd447 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0cb5f3f8049d486f8829082f65a2e785'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:23.709631 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1199/2405] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 04:11:23 2022] DELETE /identity/v3/users/0cb5f3f8049d486f8829082f65a2e785 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:11:23.984536 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d9ef7f-adbd-4202-966b-13091a07763b admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:23.986254 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-32d9ef7f-adbd-4202-966b-13091a07763b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:11:24.002034 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d9ef7f-adbd-4202-966b-13091a07763b admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:24.060195 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b5fa5-1238-4a33-87ad-8d3cc599cb99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:24.060893 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b5fa5-1238-4a33-87ad-8d3cc599cb99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:24.061098 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b5fa5-1238-4a33-87ad-8d3cc599cb99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:24.061371 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b5fa5-1238-4a33-87ad-8d3cc599cb99 admin admin] PATH_INFO: `/v3/projects/48d692e270a441f08950c581e2a0d9d8` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:24.062491 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0b5fa5-1238-4a33-87ad-8d3cc599cb99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48d692e270a441f08950c581e2a0d9d8)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:24.064472 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0b5fa5-1238-4a33-87ad-8d3cc599cb99 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:24.109875 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-6e0b5fa5-1238-4a33-87ad-8d3cc599cb99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48d692e270a441f08950c581e2a0d9d8', 'request_id': 'req-6e0b5fa5-1238-4a33-87ad-8d3cc599cb99'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:24.115464 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-6e0b5fa5-1238-4a33-87ad-8d3cc599cb99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48d692e270a441f08950c581e2a0d9d8', 'request_id': 'req-6e0b5fa5-1238-4a33-87ad-8d3cc599cb99'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:24.121534 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-6e0b5fa5-1238-4a33-87ad-8d3cc599cb99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48d692e270a441f08950c581e2a0d9d8', 'request_id': 'req-6e0b5fa5-1238-4a33-87ad-8d3cc599cb99'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:24.134523 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-6e0b5fa5-1238-4a33-87ad-8d3cc599cb99 admin admin] The token cache is being invalidate because project 48d692e270a441f08950c581e2a0d9d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 04:11:24.134811 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-6e0b5fa5-1238-4a33-87ad-8d3cc599cb99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:24.135422 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-6e0b5fa5-1238-4a33-87ad-8d3cc599cb99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:24.136945 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1207/2406] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 04:11:23 2022] DELETE /identity/v3/projects/48d692e270a441f08950c581e2a0d9d8 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:11:24.142216 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d853ddf8-3e98-4f35-8ae3-57debcb8bdb3 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:24.142699 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d853ddf8-3e98-4f35-8ae3-57debcb8bdb3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:24.143051 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d853ddf8-3e98-4f35-8ae3-57debcb8bdb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:24.164995 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d853ddf8-3e98-4f35-8ae3-57debcb8bdb3 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:11:24.166104 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d853ddf8-3e98-4f35-8ae3-57debcb8bdb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:11:24.224996 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1200/2407] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:11:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:11:24.232638 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a29063a-d58d-4c17-b37c-4b5e175e7b8d None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:24.232922 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a29063a-d58d-4c17-b37c-4b5e175e7b8d None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:24.233192 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a29063a-d58d-4c17-b37c-4b5e175e7b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:24.254113 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-4a29063a-d58d-4c17-b37c-4b5e175e7b8d None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:11:24.255306 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-4a29063a-d58d-4c17-b37c-4b5e175e7b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:11:24.264320 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1208/2408] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:11:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 04:11:24.268874 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d853ddf8-3e98-4f35-8ae3-57debcb8bdb3 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:24.279246 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d853ddf8-3e98-4f35-8ae3-57debcb8bdb3 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:24.283875 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d139f4-896e-4583-8ba6-41ac66797f18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:24.284501 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d139f4-896e-4583-8ba6-41ac66797f18 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:24.284778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d139f4-896e-4583-8ba6-41ac66797f18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:24.285047 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d139f4-896e-4583-8ba6-41ac66797f18 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:24.285681 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d139f4-896e-4583-8ba6-41ac66797f18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 04:11:24.285979 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d139f4-896e-4583-8ba6-41ac66797f18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:24.287562 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d139f4-896e-4583-8ba6-41ac66797f18 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:24.298898 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1201/2409] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 04:11:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 04:11:24.303242 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a29063a-d58d-4c17-b37c-4b5e175e7b8d None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:24.312808 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a29063a-d58d-4c17-b37c-4b5e175e7b8d None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:24.317501 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16cbd1f-a47c-46e5-9ee5-49bb31b8189b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:24.318095 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16cbd1f-a47c-46e5-9ee5-49bb31b8189b admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:24.318374 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16cbd1f-a47c-46e5-9ee5-49bb31b8189b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:24.319179 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16cbd1f-a47c-46e5-9ee5-49bb31b8189b admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:24.319821 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16cbd1f-a47c-46e5-9ee5-49bb31b8189b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:24.321752 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16cbd1f-a47c-46e5-9ee5-49bb31b8189b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:24.338147 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:11:24.354342 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1209/2410] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 04:11:24 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:11:24.355418 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d139f4-896e-4583-8ba6-41ac66797f18 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:24.365413 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d139f4-896e-4583-8ba6-41ac66797f18 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:24.379649 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:11:24.429131 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49301697-abc7-42c4-b1d7-b2380d529468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:24.429674 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49301697-abc7-42c4-b1d7-b2380d529468 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:24.429957 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49301697-abc7-42c4-b1d7-b2380d529468 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:24.430234 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49301697-abc7-42c4-b1d7-b2380d529468 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:24.430875 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49301697-abc7-42c4-b1d7-b2380d529468 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:24.432851 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49301697-abc7-42c4-b1d7-b2380d529468 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:24.454315 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1202/2411] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 04:11:24 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 04:11:24.458302 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16cbd1f-a47c-46e5-9ee5-49bb31b8189b admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:24.469987 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16cbd1f-a47c-46e5-9ee5-49bb31b8189b admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:24.474598 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c95432-79d8-47bc-96ce-06ef9d100f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:24.475191 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c95432-79d8-47bc-96ce-06ef9d100f15 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:24.475464 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c95432-79d8-47bc-96ce-06ef9d100f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:24.475731 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c95432-79d8-47bc-96ce-06ef9d100f15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:24.476424 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c95432-79d8-47bc-96ce-06ef9d100f15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:24.477995 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c95432-79d8-47bc-96ce-06ef9d100f15 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:24.487652 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1210/2412] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:11:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:11:24.492109 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49301697-abc7-42c4-b1d7-b2380d529468 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:24.502242 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49301697-abc7-42c4-b1d7-b2380d529468 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:24.507695 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cd28f6-0b4f-4a9b-935d-da807c487360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:24.508393 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd28f6-0b4f-4a9b-935d-da807c487360 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:24.508673 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd28f6-0b4f-4a9b-935d-da807c487360 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:24.508946 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cd28f6-0b4f-4a9b-935d-da807c487360 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:24.509610 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cd28f6-0b4f-4a9b-935d-da807c487360 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:24.511213 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cd28f6-0b4f-4a9b-935d-da807c487360 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:24.521737 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1203/2413] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:11:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:11:24.525709 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c95432-79d8-47bc-96ce-06ef9d100f15 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:24.535835 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c95432-79d8-47bc-96ce-06ef9d100f15 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:24.540756 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0a8983-9f77-481f-9e36-abbffbe8ca74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:24.541402 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a8983-9f77-481f-9e36-abbffbe8ca74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:24.541704 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a8983-9f77-481f-9e36-abbffbe8ca74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:24.541975 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0a8983-9f77-481f-9e36-abbffbe8ca74 admin admin] PATH_INFO: `/v3/projects/4d2280ad80f94d8eabea73880a31b9ea/users/4300415132d5477abd0cd93081a5622c/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:24.557705 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0a8983-9f77-481f-9e36-abbffbe8ca74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d2280ad80f94d8eabea73880a31b9ea, user_id=4300415132d5477abd0cd93081a5622c, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:24.560387 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0a8983-9f77-481f-9e36-abbffbe8ca74 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:24.573451 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1211/2414] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 04:11:24 2022] PUT /identity/v3/projects/4d2280ad80f94d8eabea73880a31b9ea/users/4300415132d5477abd0cd93081a5622c/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 04:11:24.583850 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81136ada-7c6b-44bd-bdb6-c6af11bcef0a None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:24.584173 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81136ada-7c6b-44bd-bdb6-c6af11bcef0a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:24.584454 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81136ada-7c6b-44bd-bdb6-c6af11bcef0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:24.611577 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-81136ada-7c6b-44bd-bdb6-c6af11bcef0a None None] MFA Rules not processed for user `4300415132d5477abd0cd93081a5622c`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:11:24.612816 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-81136ada-7c6b-44bd-bdb6-c6af11bcef0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:11:24.676530 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1204/2415] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:11:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:11:24.681556 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0a8983-9f77-481f-9e36-abbffbe8ca74 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:24.743085 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0a8983-9f77-481f-9e36-abbffbe8ca74 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:24.753478 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2b36a9-0dcf-40c9-ae93-3dc997bcd32c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:24.754091 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b36a9-0dcf-40c9-ae93-3dc997bcd32c admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:24.754367 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b36a9-0dcf-40c9-ae93-3dc997bcd32c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:24.754636 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2b36a9-0dcf-40c9-ae93-3dc997bcd32c admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:24.755280 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2b36a9-0dcf-40c9-ae93-3dc997bcd32c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:24.757258 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2b36a9-0dcf-40c9-ae93-3dc997bcd32c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:24.785151 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1212/2416] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 04:11:24 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:11:24.790108 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81136ada-7c6b-44bd-bdb6-c6af11bcef0a None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:24.803349 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81136ada-7c6b-44bd-bdb6-c6af11bcef0a None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:24.860758 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33c2cc7-7acb-4061-b1e8-1f8954a89bea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:24.861373 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c2cc7-7acb-4061-b1e8-1f8954a89bea admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:24.861652 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c2cc7-7acb-4061-b1e8-1f8954a89bea admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:24.861926 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c2cc7-7acb-4061-b1e8-1f8954a89bea admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:24.862556 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33c2cc7-7acb-4061-b1e8-1f8954a89bea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:24.864496 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33c2cc7-7acb-4061-b1e8-1f8954a89bea admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:24.885415 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1205/2417] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 04:11:24 2022] POST /identity/v3/users => generated 299 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 04:11:24.889492 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2b36a9-0dcf-40c9-ae93-3dc997bcd32c admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:24.900283 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2b36a9-0dcf-40c9-ae93-3dc997bcd32c admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:24.904836 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f643d773-616d-4c91-a6e1-3a98a5af1ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:24.905426 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f643d773-616d-4c91-a6e1-3a98a5af1ee8 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:24.905702 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f643d773-616d-4c91-a6e1-3a98a5af1ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:24.905970 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f643d773-616d-4c91-a6e1-3a98a5af1ee8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:24.906601 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f643d773-616d-4c91-a6e1-3a98a5af1ee8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:24.908615 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f643d773-616d-4c91-a6e1-3a98a5af1ee8 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:24.919185 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1213/2418] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:11:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:11:24.923137 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33c2cc7-7acb-4061-b1e8-1f8954a89bea admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:24.933055 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33c2cc7-7acb-4061-b1e8-1f8954a89bea admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:24.937536 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffd32c7-1530-4bde-9d3d-697f4328dd90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:24.938182 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd32c7-1530-4bde-9d3d-697f4328dd90 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:24.938459 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd32c7-1530-4bde-9d3d-697f4328dd90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:24.938755 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd32c7-1530-4bde-9d3d-697f4328dd90 admin admin] PATH_INFO: `/v3/projects/6f5e6937a2794530b291a3f30bb85a22/users/67bde92c89344607be711dff7196cc75/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:24.952275 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:11:24.955548 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffd32c7-1530-4bde-9d3d-697f4328dd90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f5e6937a2794530b291a3f30bb85a22, user_id=67bde92c89344607be711dff7196cc75, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:24.958691 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ffd32c7-1530-4bde-9d3d-697f4328dd90 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:24.972250 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1206/2419] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 04:11:24 2022] PUT /identity/v3/projects/6f5e6937a2794530b291a3f30bb85a22/users/67bde92c89344607be711dff7196cc75/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:11:24.976068 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f643d773-616d-4c91-a6e1-3a98a5af1ee8 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:24.985936 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f643d773-616d-4c91-a6e1-3a98a5af1ee8 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:25.042951 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7c992e-e12a-49cf-9eb0-7bfb0d3c304e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:25.043550 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7c992e-e12a-49cf-9eb0-7bfb0d3c304e admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:25.043825 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7c992e-e12a-49cf-9eb0-7bfb0d3c304e admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:25.044129 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7c992e-e12a-49cf-9eb0-7bfb0d3c304e admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:25.044758 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7c992e-e12a-49cf-9eb0-7bfb0d3c304e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:25.046320 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7c992e-e12a-49cf-9eb0-7bfb0d3c304e admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:25.056005 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1214/2420] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:11:24 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:11:25.060272 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffd32c7-1530-4bde-9d3d-697f4328dd90 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:25.070828 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffd32c7-1530-4bde-9d3d-697f4328dd90 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:25.075371 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9789ec2e-af72-4f87-bb0b-92de7b68d92d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:25.076055 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9789ec2e-af72-4f87-bb0b-92de7b68d92d admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:25.076338 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9789ec2e-af72-4f87-bb0b-92de7b68d92d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:25.076604 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9789ec2e-af72-4f87-bb0b-92de7b68d92d admin admin] PATH_INFO: `/v3/projects/6f5e6937a2794530b291a3f30bb85a22/users/67bde92c89344607be711dff7196cc75/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:25.078512 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9789ec2e-af72-4f87-bb0b-92de7b68d92d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f5e6937a2794530b291a3f30bb85a22, user_id=67bde92c89344607be711dff7196cc75, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:25.081148 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9789ec2e-af72-4f87-bb0b-92de7b68d92d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:25.091492 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1207/2421] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 04:11:25 2022] PUT /identity/v3/projects/6f5e6937a2794530b291a3f30bb85a22/users/67bde92c89344607be711dff7196cc75/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:11:25.102295 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c4948c-7942-42c6-9448-1f2ad0203a74 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:25.102773 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c4948c-7942-42c6-9448-1f2ad0203a74 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:25.102864 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c4948c-7942-42c6-9448-1f2ad0203a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:25.129943 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-18c4948c-7942-42c6-9448-1f2ad0203a74 None None] MFA Rules not processed for user `67bde92c89344607be711dff7196cc75`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:11:25.131655 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-18c4948c-7942-42c6-9448-1f2ad0203a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:11:25.148135 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:11:25.199737 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1215/2422] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:11:25 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:11:25.210997 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9789ec2e-af72-4f87-bb0b-92de7b68d92d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:25.212798 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9789ec2e-af72-4f87-bb0b-92de7b68d92d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:11:25.269639 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9789ec2e-af72-4f87-bb0b-92de7b68d92d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:25.273215 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c9be99-3839-4aa6-a763-d52bc6e9ddd8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:25.273804 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c9be99-3839-4aa6-a763-d52bc6e9ddd8 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:25.274077 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c9be99-3839-4aa6-a763-d52bc6e9ddd8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:25.274342 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c9be99-3839-4aa6-a763-d52bc6e9ddd8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:25.283107 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c9be99-3839-4aa6-a763-d52bc6e9ddd8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:25.284914 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c9be99-3839-4aa6-a763-d52bc6e9ddd8 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:25.294060 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1208/2423] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:11:25 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:11:25.834660 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c4948c-7942-42c6-9448-1f2ad0203a74 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:25.897982 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c4948c-7942-42c6-9448-1f2ad0203a74 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:25.902578 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec18283-4381-4aeb-b99c-6ac3f6add198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:25.903202 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec18283-4381-4aeb-b99c-6ac3f6add198 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:25.903508 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec18283-4381-4aeb-b99c-6ac3f6add198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:25.903802 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec18283-4381-4aeb-b99c-6ac3f6add198 admin admin] PATH_INFO: `/v3/users/4300415132d5477abd0cd93081a5622c` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:25.904975 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec18283-4381-4aeb-b99c-6ac3f6add198 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4300415132d5477abd0cd93081a5622c)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:25.906871 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec18283-4381-4aeb-b99c-6ac3f6add198 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:25.951867 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-bec18283-4381-4aeb-b99c-6ac3f6add198 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4300415132d5477abd0cd93081a5622c'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:25.952531 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-bec18283-4381-4aeb-b99c-6ac3f6add198 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4300415132d5477abd0cd93081a5622c'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:25.953130 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-bec18283-4381-4aeb-b99c-6ac3f6add198 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4300415132d5477abd0cd93081a5622c'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:25.965196 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-bec18283-4381-4aeb-b99c-6ac3f6add198 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4300415132d5477abd0cd93081a5622c'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:25.972292 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-bec18283-4381-4aeb-b99c-6ac3f6add198 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4300415132d5477abd0cd93081a5622c'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:25.984713 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1216/2424] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 04:11:25 2022] DELETE /identity/v3/users/4300415132d5477abd0cd93081a5622c => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:11:25.989194 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e6bae9-6ec5-42ec-9cbc-44dabd32626d None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:25.989480 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e6bae9-6ec5-42ec-9cbc-44dabd32626d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:25.989749 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e6bae9-6ec5-42ec-9cbc-44dabd32626d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:26.010098 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-36e6bae9-6ec5-42ec-9cbc-44dabd32626d None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:11:26.011279 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-36e6bae9-6ec5-42ec-9cbc-44dabd32626d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:11:26.068189 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1209/2425] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:11:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:11:26.078632 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec18283-4381-4aeb-b99c-6ac3f6add198 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:26.080062 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-bec18283-4381-4aeb-b99c-6ac3f6add198 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:11:26.114749 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:11:26.138836 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec18283-4381-4aeb-b99c-6ac3f6add198 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:26.142538 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b64d921-783d-4ece-aab6-3334e278bc50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:26.143133 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b64d921-783d-4ece-aab6-3334e278bc50 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:26.143411 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b64d921-783d-4ece-aab6-3334e278bc50 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:26.143686 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b64d921-783d-4ece-aab6-3334e278bc50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:26.152740 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b64d921-783d-4ece-aab6-3334e278bc50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:26.154191 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b64d921-783d-4ece-aab6-3334e278bc50 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:26.163603 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1217/2426] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:11:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:11:26.427149 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e6bae9-6ec5-42ec-9cbc-44dabd32626d None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:26.429260 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-36e6bae9-6ec5-42ec-9cbc-44dabd32626d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:11:26.442071 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e6bae9-6ec5-42ec-9cbc-44dabd32626d None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:26.446694 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d337fb44-b400-4032-9bfc-fdcff8915542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:26.447390 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337fb44-b400-4032-9bfc-fdcff8915542 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:26.448299 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337fb44-b400-4032-9bfc-fdcff8915542 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:26.448993 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d337fb44-b400-4032-9bfc-fdcff8915542 admin admin] PATH_INFO: `/v3/projects/4d2280ad80f94d8eabea73880a31b9ea` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:26.450040 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d337fb44-b400-4032-9bfc-fdcff8915542 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d2280ad80f94d8eabea73880a31b9ea)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:26.452064 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d337fb44-b400-4032-9bfc-fdcff8915542 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:26.500432 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d337fb44-b400-4032-9bfc-fdcff8915542 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d2280ad80f94d8eabea73880a31b9ea', 'request_id': 'req-d337fb44-b400-4032-9bfc-fdcff8915542'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:26.506885 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d337fb44-b400-4032-9bfc-fdcff8915542 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d2280ad80f94d8eabea73880a31b9ea', 'request_id': 'req-d337fb44-b400-4032-9bfc-fdcff8915542'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:26.513791 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d337fb44-b400-4032-9bfc-fdcff8915542 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d2280ad80f94d8eabea73880a31b9ea', 'request_id': 'req-d337fb44-b400-4032-9bfc-fdcff8915542'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:26.524239 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d337fb44-b400-4032-9bfc-fdcff8915542 admin admin] The token cache is being invalidate because project 4d2280ad80f94d8eabea73880a31b9ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 04:11:26.524530 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d337fb44-b400-4032-9bfc-fdcff8915542 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:26.525184 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d337fb44-b400-4032-9bfc-fdcff8915542 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:26.526921 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1210/2427] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 04:11:26 2022] DELETE /identity/v3/projects/4d2280ad80f94d8eabea73880a31b9ea => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:11:26.530901 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b64d921-783d-4ece-aab6-3334e278bc50 service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:26.541300 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b64d921-783d-4ece-aab6-3334e278bc50 service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:26.598118 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9b8bb3-f5df-4cdf-8cc8-839b27a0331b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:26.598743 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b8bb3-f5df-4cdf-8cc8-839b27a0331b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:26.599016 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b8bb3-f5df-4cdf-8cc8-839b27a0331b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:26.599287 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9b8bb3-f5df-4cdf-8cc8-839b27a0331b admin admin] PATH_INFO: `/v3/users/67bde92c89344607be711dff7196cc75` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:26.600933 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9b8bb3-f5df-4cdf-8cc8-839b27a0331b admin admin] RBAC: Authorizing `identity:delete_user(user_id=67bde92c89344607be711dff7196cc75)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:26.603149 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9b8bb3-f5df-4cdf-8cc8-839b27a0331b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:26.647980 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ee9b8bb3-f5df-4cdf-8cc8-839b27a0331b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67bde92c89344607be711dff7196cc75'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:26.648623 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ee9b8bb3-f5df-4cdf-8cc8-839b27a0331b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67bde92c89344607be711dff7196cc75'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:26.649219 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ee9b8bb3-f5df-4cdf-8cc8-839b27a0331b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67bde92c89344607be711dff7196cc75'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:26.661726 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ee9b8bb3-f5df-4cdf-8cc8-839b27a0331b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67bde92c89344607be711dff7196cc75'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:26.669195 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ee9b8bb3-f5df-4cdf-8cc8-839b27a0331b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67bde92c89344607be711dff7196cc75'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:26.681779 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1218/2428] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 04:11:26 2022] DELETE /identity/v3/users/67bde92c89344607be711dff7196cc75 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 04:11:26.948765 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d337fb44-b400-4032-9bfc-fdcff8915542 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:26.950324 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d337fb44-b400-4032-9bfc-fdcff8915542 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:11:26.962728 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:11:26.967476 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d337fb44-b400-4032-9bfc-fdcff8915542 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:27.028623 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10d38d6-2cdb-465f-aaa2-ebd6e325b0d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:27.029184 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10d38d6-2cdb-465f-aaa2-ebd6e325b0d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:27.029465 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10d38d6-2cdb-465f-aaa2-ebd6e325b0d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:27.029744 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10d38d6-2cdb-465f-aaa2-ebd6e325b0d1 admin admin] PATH_INFO: `/v3/projects/6f5e6937a2794530b291a3f30bb85a22` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:27.030866 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10d38d6-2cdb-465f-aaa2-ebd6e325b0d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f5e6937a2794530b291a3f30bb85a22)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:27.032797 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10d38d6-2cdb-465f-aaa2-ebd6e325b0d1 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:27.078484 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f10d38d6-2cdb-465f-aaa2-ebd6e325b0d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f5e6937a2794530b291a3f30bb85a22', 'request_id': 'req-f10d38d6-2cdb-465f-aaa2-ebd6e325b0d1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:27.084991 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f10d38d6-2cdb-465f-aaa2-ebd6e325b0d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f5e6937a2794530b291a3f30bb85a22', 'request_id': 'req-f10d38d6-2cdb-465f-aaa2-ebd6e325b0d1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:27.091963 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f10d38d6-2cdb-465f-aaa2-ebd6e325b0d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f5e6937a2794530b291a3f30bb85a22', 'request_id': 'req-f10d38d6-2cdb-465f-aaa2-ebd6e325b0d1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:27.102118 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f10d38d6-2cdb-465f-aaa2-ebd6e325b0d1 admin admin] The token cache is being invalidate because project 6f5e6937a2794530b291a3f30bb85a22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 04:11:27.102404 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f10d38d6-2cdb-465f-aaa2-ebd6e325b0d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:27.103020 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f10d38d6-2cdb-465f-aaa2-ebd6e325b0d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:11:27.104536 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1211/2429] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 04:11:26 2022] DELETE /identity/v3/projects/6f5e6937a2794530b291a3f30bb85a22 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 04:11:27.110357 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b0a2d-4fd5-49f8-baed-97697f049427 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:27.110646 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b0a2d-4fd5-49f8-baed-97697f049427 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:27.110925 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8b0a2d-4fd5-49f8-baed-97697f049427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:27.132210 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-8d8b0a2d-4fd5-49f8-baed-97697f049427 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:11:27.133411 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-8d8b0a2d-4fd5-49f8-baed-97697f049427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:11:27.182023 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:11:27.194972 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1219/2430] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:11:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:11:27.202971 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c165d1-9749-49bf-b290-ccb0bae9340c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:27.203259 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c165d1-9749-49bf-b290-ccb0bae9340c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:27.203780 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c165d1-9749-49bf-b290-ccb0bae9340c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:27.223876 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-06c165d1-9749-49bf-b290-ccb0bae9340c None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:11:27.225104 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-06c165d1-9749-49bf-b290-ccb0bae9340c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:11:27.232935 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1212/2431] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:11:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:11:27.236988 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8b0a2d-4fd5-49f8-baed-97697f049427 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:27.246496 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8b0a2d-4fd5-49f8-baed-97697f049427 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:27.252012 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eadf90-f34e-47cb-8090-1514f647ac4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:27.252621 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eadf90-f34e-47cb-8090-1514f647ac4f admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:27.252897 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eadf90-f34e-47cb-8090-1514f647ac4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:27.253162 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eadf90-f34e-47cb-8090-1514f647ac4f admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:27.253828 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eadf90-f34e-47cb-8090-1514f647ac4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 04:11:27.254125 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eadf90-f34e-47cb-8090-1514f647ac4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:27.256403 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4eadf90-f34e-47cb-8090-1514f647ac4f admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:27.267986 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1220/2432] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 04:11:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 04:11:27.272751 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c165d1-9749-49bf-b290-ccb0bae9340c None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:27.281964 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c165d1-9749-49bf-b290-ccb0bae9340c None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:27.286568 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921fd99f-7442-4f0e-9b9e-435eacf57617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:27.287162 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921fd99f-7442-4f0e-9b9e-435eacf57617 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:27.287440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921fd99f-7442-4f0e-9b9e-435eacf57617 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:27.287711 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921fd99f-7442-4f0e-9b9e-435eacf57617 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:27.288383 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921fd99f-7442-4f0e-9b9e-435eacf57617 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:27.290260 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921fd99f-7442-4f0e-9b9e-435eacf57617 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:27.317419 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1213/2433] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 04:11:27 2022] POST /identity/v3/projects => generated 369 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:11:27.321390 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eadf90-f34e-47cb-8090-1514f647ac4f admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:27.331065 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eadf90-f34e-47cb-8090-1514f647ac4f admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:27.386549 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608586d4-a908-4e89-b3cb-f9623feafb9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:27.387138 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608586d4-a908-4e89-b3cb-f9623feafb9b admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:27.387414 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608586d4-a908-4e89-b3cb-f9623feafb9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:27.387680 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608586d4-a908-4e89-b3cb-f9623feafb9b admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:27.388337 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608586d4-a908-4e89-b3cb-f9623feafb9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:27.390221 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608586d4-a908-4e89-b3cb-f9623feafb9b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:27.411699 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1221/2434] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 04:11:27 2022] POST /identity/v3/users => generated 285 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:11:27.415796 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921fd99f-7442-4f0e-9b9e-435eacf57617 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:27.425959 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921fd99f-7442-4f0e-9b9e-435eacf57617 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:27.430543 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472318dc-46f4-4481-95dd-80dc19905b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:27.431140 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472318dc-46f4-4481-95dd-80dc19905b4f admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:27.431415 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472318dc-46f4-4481-95dd-80dc19905b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:27.431686 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472318dc-46f4-4481-95dd-80dc19905b4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:27.432353 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472318dc-46f4-4481-95dd-80dc19905b4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:27.433927 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472318dc-46f4-4481-95dd-80dc19905b4f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:27.444108 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1214/2435] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:11:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:11:27.447922 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608586d4-a908-4e89-b3cb-f9623feafb9b admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:27.457797 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608586d4-a908-4e89-b3cb-f9623feafb9b admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:27.463164 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21c8da3-4000-4e2b-b027-082842317314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:27.463762 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21c8da3-4000-4e2b-b027-082842317314 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:27.464071 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21c8da3-4000-4e2b-b027-082842317314 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:27.464346 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21c8da3-4000-4e2b-b027-082842317314 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:27.465006 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21c8da3-4000-4e2b-b027-082842317314 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:27.466570 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21c8da3-4000-4e2b-b027-082842317314 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:27.476940 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1222/2436] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:11:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:11:27.480974 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472318dc-46f4-4481-95dd-80dc19905b4f admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:27.491298 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472318dc-46f4-4481-95dd-80dc19905b4f admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:27.495904 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23702152-d4c4-457c-bf32-b341d5811fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:27.496575 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23702152-d4c4-457c-bf32-b341d5811fc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:27.496854 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23702152-d4c4-457c-bf32-b341d5811fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:27.497128 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23702152-d4c4-457c-bf32-b341d5811fc4 admin admin] PATH_INFO: `/v3/projects/1ea7c4292d714d88acdc30de1ad2831a/users/b417cbe4b7fd4d219a183e1d5475d13e/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:27.512270 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23702152-d4c4-457c-bf32-b341d5811fc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ea7c4292d714d88acdc30de1ad2831a, user_id=b417cbe4b7fd4d219a183e1d5475d13e, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:27.514925 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23702152-d4c4-457c-bf32-b341d5811fc4 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:27.527623 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1215/2437] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 04:11:27 2022] PUT /identity/v3/projects/1ea7c4292d714d88acdc30de1ad2831a/users/b417cbe4b7fd4d219a183e1d5475d13e/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:11:27.537917 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21295db6-6334-44b0-9a1f-6e9497498e98 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:27.538169 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21295db6-6334-44b0-9a1f-6e9497498e98 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:27.538521 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21295db6-6334-44b0-9a1f-6e9497498e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:27.567619 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-21295db6-6334-44b0-9a1f-6e9497498e98 None None] MFA Rules not processed for user `b417cbe4b7fd4d219a183e1d5475d13e`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:11:27.569475 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-21295db6-6334-44b0-9a1f-6e9497498e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:11:27.639284 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1223/2438] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:11:27 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:11:27.651394 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23702152-d4c4-457c-bf32-b341d5811fc4 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:11:27.653338 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-23702152-d4c4-457c-bf32-b341d5811fc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:11:27.687283 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:11:27.711796 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23702152-d4c4-457c-bf32-b341d5811fc4 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:11:27.715600 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d72f68-b70e-4a6e-ae68-8d663cb28909 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '50aab624ee9c4651a7cd42124bcc465f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:11:27.716255 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d72f68-b70e-4a6e-ae68-8d663cb28909 service glance] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:11:27.716537 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d72f68-b70e-4a6e-ae68-8d663cb28909 service glance] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:11:27.716810 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d72f68-b70e-4a6e-ae68-8d663cb28909 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:11:27.725254 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d72f68-b70e-4a6e-ae68-8d663cb28909 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:11:27.726663 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d72f68-b70e-4a6e-ae68-8d663cb28909 service glance] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:11:27.735416 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1216/2439] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:11:27 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:11:37.636499 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=74,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:11:37.731375 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=12 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:12:02.119534 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21295db6-6334-44b0-9a1f-6e9497498e98 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:12:02.182070 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21295db6-6334-44b0-9a1f-6e9497498e98 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:12:02.186592 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a900d998-8664-486d-be4d-9339081202b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:12:02.187223 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a900d998-8664-486d-be4d-9339081202b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:12:02.187500 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a900d998-8664-486d-be4d-9339081202b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:12:02.187769 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a900d998-8664-486d-be4d-9339081202b6 admin admin] PATH_INFO: `/v3/users/b417cbe4b7fd4d219a183e1d5475d13e` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:12:02.188917 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a900d998-8664-486d-be4d-9339081202b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b417cbe4b7fd4d219a183e1d5475d13e)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:12:02.190834 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a900d998-8664-486d-be4d-9339081202b6 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:12:02.237836 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-a900d998-8664-486d-be4d-9339081202b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b417cbe4b7fd4d219a183e1d5475d13e'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:12:02.238487 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-a900d998-8664-486d-be4d-9339081202b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b417cbe4b7fd4d219a183e1d5475d13e'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:12:02.239095 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-a900d998-8664-486d-be4d-9339081202b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b417cbe4b7fd4d219a183e1d5475d13e'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:12:02.252051 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-a900d998-8664-486d-be4d-9339081202b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b417cbe4b7fd4d219a183e1d5475d13e'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:12:02.260122 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-a900d998-8664-486d-be4d-9339081202b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b417cbe4b7fd4d219a183e1d5475d13e'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:12:02.273126 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1224/2440] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 04:12:02 2022] DELETE /identity/v3/users/b417cbe4b7fd4d219a183e1d5475d13e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:12:02.277656 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c3597-58f9-4442-b919-16dbf95c57fd None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:12:02.278140 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c3597-58f9-4442-b919-16dbf95c57fd None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:12:02.278233 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6c3597-58f9-4442-b919-16dbf95c57fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:12:02.299269 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-ca6c3597-58f9-4442-b919-16dbf95c57fd None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:12:02.300514 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ca6c3597-58f9-4442-b919-16dbf95c57fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:12:02.368724 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1217/2441] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:12:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:12:02.375871 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a900d998-8664-486d-be4d-9339081202b6 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:12:02.377320 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a900d998-8664-486d-be4d-9339081202b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:12:02.440300 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a900d998-8664-486d-be4d-9339081202b6 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:12:02.444285 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0a379c-d2c9-49ee-9b93-bde559369d82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:12:02.444932 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a379c-d2c9-49ee-9b93-bde559369d82 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:12:02.445212 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a379c-d2c9-49ee-9b93-bde559369d82 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:12:02.445512 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0a379c-d2c9-49ee-9b93-bde559369d82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:12:02.454993 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0a379c-d2c9-49ee-9b93-bde559369d82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:12:02.456498 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0a379c-d2c9-49ee-9b93-bde559369d82 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:12:02.466526 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1225/2442] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:12:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:12:02.798006 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6c3597-58f9-4442-b919-16dbf95c57fd None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:12:02.801006 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ca6c3597-58f9-4442-b919-16dbf95c57fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:12:02.815290 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6c3597-58f9-4442-b919-16dbf95c57fd None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:12:02.823969 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4acc6124-5b82-4a82-a699-20171a3be052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:12:02.824537 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acc6124-5b82-4a82-a699-20171a3be052 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:12:02.824814 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acc6124-5b82-4a82-a699-20171a3be052 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:12:02.825178 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4acc6124-5b82-4a82-a699-20171a3be052 admin admin] PATH_INFO: `/v3/projects/1ea7c4292d714d88acdc30de1ad2831a` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:12:02.826205 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acc6124-5b82-4a82-a699-20171a3be052 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ea7c4292d714d88acdc30de1ad2831a)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:12:02.829156 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4acc6124-5b82-4a82-a699-20171a3be052 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:12:02.900952 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4acc6124-5b82-4a82-a699-20171a3be052 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ea7c4292d714d88acdc30de1ad2831a', 'request_id': 'req-4acc6124-5b82-4a82-a699-20171a3be052'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:12:02.908733 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4acc6124-5b82-4a82-a699-20171a3be052 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ea7c4292d714d88acdc30de1ad2831a', 'request_id': 'req-4acc6124-5b82-4a82-a699-20171a3be052'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:12:02.919054 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4acc6124-5b82-4a82-a699-20171a3be052 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ea7c4292d714d88acdc30de1ad2831a', 'request_id': 'req-4acc6124-5b82-4a82-a699-20171a3be052'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:12:02.933203 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4acc6124-5b82-4a82-a699-20171a3be052 admin admin] The token cache is being invalidate because project 1ea7c4292d714d88acdc30de1ad2831a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 04:12:02.933974 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4acc6124-5b82-4a82-a699-20171a3be052 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:12:02.934781 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4acc6124-5b82-4a82-a699-20171a3be052 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:12:02.936580 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1218/2443] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 04:12:02 2022] DELETE /identity/v3/projects/1ea7c4292d714d88acdc30de1ad2831a => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:12:02.941516 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5736f4c-61a3-4735-a883-897402133ede None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:12:02.941808 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5736f4c-61a3-4735-a883-897402133ede None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:12:02.942087 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5736f4c-61a3-4735-a883-897402133ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:12:02.965104 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-d5736f4c-61a3-4735-a883-897402133ede None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:12:02.966312 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d5736f4c-61a3-4735-a883-897402133ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:12:02.986306 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:12:03.033412 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1226/2444] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:12:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:12:03.040903 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1f8ed-af91-428c-b1ee-4d3774eadc13 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:12:03.041192 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1f8ed-af91-428c-b1ee-4d3774eadc13 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:12:03.041460 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef1f8ed-af91-428c-b1ee-4d3774eadc13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:12:03.060927 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-bef1f8ed-af91-428c-b1ee-4d3774eadc13 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:12:03.062151 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-bef1f8ed-af91-428c-b1ee-4d3774eadc13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:12:03.070238 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1219/2445] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:12:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:12:03.074936 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5736f4c-61a3-4735-a883-897402133ede None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:12:03.084425 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5736f4c-61a3-4735-a883-897402133ede None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:12:03.089666 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a139440-2b9c-4ee7-8a21-77f2debfc8c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:12:03.090267 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a139440-2b9c-4ee7-8a21-77f2debfc8c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:12:03.090546 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a139440-2b9c-4ee7-8a21-77f2debfc8c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:12:03.090813 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a139440-2b9c-4ee7-8a21-77f2debfc8c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:12:03.091502 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a139440-2b9c-4ee7-8a21-77f2debfc8c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 04:12:03.091750 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a139440-2b9c-4ee7-8a21-77f2debfc8c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:12:03.093400 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a139440-2b9c-4ee7-8a21-77f2debfc8c8 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:12:03.108322 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1227/2446] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 04:12:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 04:12:03.109579 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef1f8ed-af91-428c-b1ee-4d3774eadc13 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:12:03.118837 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef1f8ed-af91-428c-b1ee-4d3774eadc13 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:12:03.123722 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a7e40-8445-4d57-a7ad-fe5a9b6bee10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:12:03.124365 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a7e40-8445-4d57-a7ad-fe5a9b6bee10 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:12:03.124818 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a7e40-8445-4d57-a7ad-fe5a9b6bee10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:12:03.124939 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739a7e40-8445-4d57-a7ad-fe5a9b6bee10 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:12:03.125527 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739a7e40-8445-4d57-a7ad-fe5a9b6bee10 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:12:03.127440 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739a7e40-8445-4d57-a7ad-fe5a9b6bee10 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:12:03.157959 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1220/2447] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 04:12:03 2022] POST /identity/v3/projects => generated 389 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:12:03.162143 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a139440-2b9c-4ee7-8a21-77f2debfc8c8 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:12:03.172031 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a139440-2b9c-4ee7-8a21-77f2debfc8c8 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:12:03.230304 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdcb4c9-bf1a-4371-aa0a-c14f134bcf8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:12:03.230935 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdcb4c9-bf1a-4371-aa0a-c14f134bcf8a admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:12:03.231218 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdcb4c9-bf1a-4371-aa0a-c14f134bcf8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:12:03.231488 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdcb4c9-bf1a-4371-aa0a-c14f134bcf8a admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:12:03.232161 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdcb4c9-bf1a-4371-aa0a-c14f134bcf8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:12:03.234070 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdcb4c9-bf1a-4371-aa0a-c14f134bcf8a admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:12:03.259293 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1228/2448] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 04:12:03 2022] POST /identity/v3/users => generated 295 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:12:03.260643 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a7e40-8445-4d57-a7ad-fe5a9b6bee10 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:12:03.270973 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739a7e40-8445-4d57-a7ad-fe5a9b6bee10 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:12:03.276161 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c02e3f3-2918-4928-88fe-eb211e82b69b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:12:03.276763 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02e3f3-2918-4928-88fe-eb211e82b69b admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:12:03.277039 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02e3f3-2918-4928-88fe-eb211e82b69b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:12:03.277306 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c02e3f3-2918-4928-88fe-eb211e82b69b admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:12:03.277967 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c02e3f3-2918-4928-88fe-eb211e82b69b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:12:03.279550 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c02e3f3-2918-4928-88fe-eb211e82b69b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:12:03.289730 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1221/2449] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:12:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:12:03.293742 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdcb4c9-bf1a-4371-aa0a-c14f134bcf8a admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:12:03.304177 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdcb4c9-bf1a-4371-aa0a-c14f134bcf8a admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:12:03.308932 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afca12b2-e11d-4b50-9f56-1cef253e4a68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:12:03.309521 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca12b2-e11d-4b50-9f56-1cef253e4a68 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:12:03.310368 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca12b2-e11d-4b50-9f56-1cef253e4a68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:12:03.310368 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca12b2-e11d-4b50-9f56-1cef253e4a68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:12:03.310698 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afca12b2-e11d-4b50-9f56-1cef253e4a68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:12:03.312332 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afca12b2-e11d-4b50-9f56-1cef253e4a68 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:12:03.325041 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1229/2450] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:12:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:12:03.328907 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c02e3f3-2918-4928-88fe-eb211e82b69b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:12:03.339441 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c02e3f3-2918-4928-88fe-eb211e82b69b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:12:03.345868 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bd7b79-5771-4336-afff-4a826df550c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:12:03.346542 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bd7b79-5771-4336-afff-4a826df550c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:12:03.346820 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bd7b79-5771-4336-afff-4a826df550c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:12:03.347091 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bd7b79-5771-4336-afff-4a826df550c0 admin admin] PATH_INFO: `/v3/projects/c64779465436419fa6e450ddf038e1c6/users/fc0ed1f72001432a821862e8902ef70b/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:12:03.364103 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bd7b79-5771-4336-afff-4a826df550c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c64779465436419fa6e450ddf038e1c6, user_id=fc0ed1f72001432a821862e8902ef70b, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:12:03.366955 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bd7b79-5771-4336-afff-4a826df550c0 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:12:03.382574 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1222/2451] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 04:12:03 2022] PUT /identity/v3/projects/c64779465436419fa6e450ddf038e1c6/users/fc0ed1f72001432a821862e8902ef70b/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:12:03.394107 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eaaac9-d166-4bbc-acdd-f1ef46cb8476 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:12:03.394394 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eaaac9-d166-4bbc-acdd-f1ef46cb8476 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:12:03.394668 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eaaac9-d166-4bbc-acdd-f1ef46cb8476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:12:03.423378 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-92eaaac9-d166-4bbc-acdd-f1ef46cb8476 None None] MFA Rules not processed for user `fc0ed1f72001432a821862e8902ef70b`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:12:03.424636 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-92eaaac9-d166-4bbc-acdd-f1ef46cb8476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:12:03.491036 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1230/2452] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:12:03 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:12:03.502586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bd7b79-5771-4336-afff-4a826df550c0 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:12:03.504379 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b4bd7b79-5771-4336-afff-4a826df550c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:12:03.527821 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:12:03.573108 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bd7b79-5771-4336-afff-4a826df550c0 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:12:03.576821 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d53dd43-20fc-4b53-bbc3-b7df8e8ff9f1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:12:03.577426 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d53dd43-20fc-4b53-bbc3-b7df8e8ff9f1 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:12:03.577707 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d53dd43-20fc-4b53-bbc3-b7df8e8ff9f1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:12:03.578121 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d53dd43-20fc-4b53-bbc3-b7df8e8ff9f1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:12:03.586537 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d53dd43-20fc-4b53-bbc3-b7df8e8ff9f1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:12:03.588098 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d53dd43-20fc-4b53-bbc3-b7df8e8ff9f1 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:12:03.596909 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1223/2453] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:12:03 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:12:09.949000 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eaaac9-d166-4bbc-acdd-f1ef46cb8476 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:12:09.950470 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-92eaaac9-d166-4bbc-acdd-f1ef46cb8476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:12:09.970443 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:12:10.012285 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eaaac9-d166-4bbc-acdd-f1ef46cb8476 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:12:10.015855 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e4689f-e53e-4c55-a55b-40bdf4b1928f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:12:10.016500 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e4689f-e53e-4c55-a55b-40bdf4b1928f service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:12:10.016779 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e4689f-e53e-4c55-a55b-40bdf4b1928f service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:12:10.017051 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e4689f-e53e-4c55-a55b-40bdf4b1928f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:12:10.018773 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-36e4689f-e53e-4c55-a55b-40bdf4b1928f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:12:10.080341 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e4689f-e53e-4c55-a55b-40bdf4b1928f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:12:10.081707 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e4689f-e53e-4c55-a55b-40bdf4b1928f service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:12:10.091198 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1231/2454] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:12:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:12:13.129837 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d53dd43-20fc-4b53-bbc3-b7df8e8ff9f1 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:12:13.132259 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6d53dd43-20fc-4b53-bbc3-b7df8e8ff9f1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:12:13.217566 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d53dd43-20fc-4b53-bbc3-b7df8e8ff9f1 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:12:13.222635 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a4cddc-b0ee-4da1-8a84-eccd7257be3a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4b9bd741574ddabadd4b7651ebc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:12:13.223249 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a4cddc-b0ee-4da1-8a84-eccd7257be3a service placement] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:12:13.223532 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a4cddc-b0ee-4da1-8a84-eccd7257be3a service placement] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:12:13.223810 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a4cddc-b0ee-4da1-8a84-eccd7257be3a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:12:13.228697 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f9a4cddc-b0ee-4da1-8a84-eccd7257be3a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:12:13.241014 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a4cddc-b0ee-4da1-8a84-eccd7257be3a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:12:13.243096 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a4cddc-b0ee-4da1-8a84-eccd7257be3a service placement] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:12:13.253605 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1224/2455] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:12:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:12:16.363319 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e4689f-e53e-4c55-a55b-40bdf4b1928f service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:12:16.365377 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-36e4689f-e53e-4c55-a55b-40bdf4b1928f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:12:16.380693 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e4689f-e53e-4c55-a55b-40bdf4b1928f service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:12:16.386181 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc36c18-c8c7-43fc-8691-6204eb27a42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:12:16.386828 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc36c18-c8c7-43fc-8691-6204eb27a42e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:12:16.387161 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc36c18-c8c7-43fc-8691-6204eb27a42e admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:12:16.387386 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc36c18-c8c7-43fc-8691-6204eb27a42e admin admin] PATH_INFO: `/v3/users/7f89c4e2fa8c495fa8bc7f64a3635fe0` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:12:16.388540 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc36c18-c8c7-43fc-8691-6204eb27a42e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f89c4e2fa8c495fa8bc7f64a3635fe0)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:12:16.390456 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc36c18-c8c7-43fc-8691-6204eb27a42e admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:12:16.436010 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-7cc36c18-c8c7-43fc-8691-6204eb27a42e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f89c4e2fa8c495fa8bc7f64a3635fe0'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:12:16.436675 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-7cc36c18-c8c7-43fc-8691-6204eb27a42e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f89c4e2fa8c495fa8bc7f64a3635fe0'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:12:16.437408 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-7cc36c18-c8c7-43fc-8691-6204eb27a42e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f89c4e2fa8c495fa8bc7f64a3635fe0'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:12:16.451240 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-7cc36c18-c8c7-43fc-8691-6204eb27a42e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f89c4e2fa8c495fa8bc7f64a3635fe0'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:12:16.459231 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-7cc36c18-c8c7-43fc-8691-6204eb27a42e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f89c4e2fa8c495fa8bc7f64a3635fe0'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:12:16.474600 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1232/2456] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 04:12:16 2022] DELETE /identity/v3/users/7f89c4e2fa8c495fa8bc7f64a3635fe0 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:12:16.651342 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a4cddc-b0ee-4da1-8a84-eccd7257be3a service placement] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:12:16.666495 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a4cddc-b0ee-4da1-8a84-eccd7257be3a service placement] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:12:16.723110 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c64faa0-e72e-4c94-8a96-c1a6d2e4909f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:12:16.723733 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c64faa0-e72e-4c94-8a96-c1a6d2e4909f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:12:16.724038 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c64faa0-e72e-4c94-8a96-c1a6d2e4909f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:12:16.724310 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c64faa0-e72e-4c94-8a96-c1a6d2e4909f admin admin] PATH_INFO: `/v3/projects/3e61d1b98c0b474a95a99962af8b20b5` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:12:16.725405 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c64faa0-e72e-4c94-8a96-c1a6d2e4909f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e61d1b98c0b474a95a99962af8b20b5)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:12:16.727805 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c64faa0-e72e-4c94-8a96-c1a6d2e4909f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:12:16.774273 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-3c64faa0-e72e-4c94-8a96-c1a6d2e4909f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e61d1b98c0b474a95a99962af8b20b5', 'request_id': 'req-3c64faa0-e72e-4c94-8a96-c1a6d2e4909f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:12:16.780432 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-3c64faa0-e72e-4c94-8a96-c1a6d2e4909f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e61d1b98c0b474a95a99962af8b20b5', 'request_id': 'req-3c64faa0-e72e-4c94-8a96-c1a6d2e4909f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:12:16.782097 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:12:16.792072 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-3c64faa0-e72e-4c94-8a96-c1a6d2e4909f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e61d1b98c0b474a95a99962af8b20b5', 'request_id': 'req-3c64faa0-e72e-4c94-8a96-c1a6d2e4909f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:12:16.802314 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-3c64faa0-e72e-4c94-8a96-c1a6d2e4909f admin admin] The token cache is being invalidate because project 3e61d1b98c0b474a95a99962af8b20b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 04:12:16.802605 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-3c64faa0-e72e-4c94-8a96-c1a6d2e4909f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:12:16.803247 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-3c64faa0-e72e-4c94-8a96-c1a6d2e4909f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:12:16.805329 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1225/2457] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 04:12:16 2022] DELETE /identity/v3/projects/3e61d1b98c0b474a95a99962af8b20b5 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:12:16.809305 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc36c18-c8c7-43fc-8691-6204eb27a42e admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:12:16.810740 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-7cc36c18-c8c7-43fc-8691-6204eb27a42e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:12:16.823805 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc36c18-c8c7-43fc-8691-6204eb27a42e admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:12:16.860949 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:12:16.881405 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a026a4cb-1595-448c-a65c-9f1cf364a5c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:12:16.882048 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a026a4cb-1595-448c-a65c-9f1cf364a5c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:12:16.882324 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a026a4cb-1595-448c-a65c-9f1cf364a5c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:12:16.882596 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a026a4cb-1595-448c-a65c-9f1cf364a5c0 admin admin] PATH_INFO: `/v3/users/18c636cd388b4d6cbf77f9c97d3f5a76` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:12:16.883761 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a026a4cb-1595-448c-a65c-9f1cf364a5c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=18c636cd388b4d6cbf77f9c97d3f5a76)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:12:16.885700 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a026a4cb-1595-448c-a65c-9f1cf364a5c0 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:12:16.931764 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-a026a4cb-1595-448c-a65c-9f1cf364a5c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '18c636cd388b4d6cbf77f9c97d3f5a76'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:12:16.932376 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-a026a4cb-1595-448c-a65c-9f1cf364a5c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '18c636cd388b4d6cbf77f9c97d3f5a76'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:12:16.932982 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-a026a4cb-1595-448c-a65c-9f1cf364a5c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '18c636cd388b4d6cbf77f9c97d3f5a76'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:12:16.945341 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-a026a4cb-1595-448c-a65c-9f1cf364a5c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '18c636cd388b4d6cbf77f9c97d3f5a76'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:12:16.952236 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-a026a4cb-1595-448c-a65c-9f1cf364a5c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '18c636cd388b4d6cbf77f9c97d3f5a76'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:12:16.964735 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1233/2458] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 04:12:16 2022] DELETE /identity/v3/users/18c636cd388b4d6cbf77f9c97d3f5a76 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:12:17.132171 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c64faa0-e72e-4c94-8a96-c1a6d2e4909f admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:12:17.133697 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3c64faa0-e72e-4c94-8a96-c1a6d2e4909f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:12:17.150509 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c64faa0-e72e-4c94-8a96-c1a6d2e4909f admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:12:17.210178 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaee646-cb8a-403c-9d4b-a46fdfe520a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:12:17.210814 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaee646-cb8a-403c-9d4b-a46fdfe520a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:12:17.211637 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaee646-cb8a-403c-9d4b-a46fdfe520a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:12:17.211913 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaee646-cb8a-403c-9d4b-a46fdfe520a9 admin admin] PATH_INFO: `/v3/projects/01996d414de340c39117fd2c347da57c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:12:17.213063 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaee646-cb8a-403c-9d4b-a46fdfe520a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01996d414de340c39117fd2c347da57c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:12:17.214967 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaee646-cb8a-403c-9d4b-a46fdfe520a9 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:12:17.261363 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7aaee646-cb8a-403c-9d4b-a46fdfe520a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01996d414de340c39117fd2c347da57c', 'request_id': 'req-7aaee646-cb8a-403c-9d4b-a46fdfe520a9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:12:17.267442 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7aaee646-cb8a-403c-9d4b-a46fdfe520a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01996d414de340c39117fd2c347da57c', 'request_id': 'req-7aaee646-cb8a-403c-9d4b-a46fdfe520a9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:12:17.274980 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7aaee646-cb8a-403c-9d4b-a46fdfe520a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01996d414de340c39117fd2c347da57c', 'request_id': 'req-7aaee646-cb8a-403c-9d4b-a46fdfe520a9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:12:17.285497 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7aaee646-cb8a-403c-9d4b-a46fdfe520a9 admin admin] The token cache is being invalidate because project 01996d414de340c39117fd2c347da57c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 04:12:17.285784 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7aaee646-cb8a-403c-9d4b-a46fdfe520a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:12:17.286409 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7aaee646-cb8a-403c-9d4b-a46fdfe520a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:12:17.287889 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1226/2459] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 04:12:17 2022] DELETE /identity/v3/projects/01996d414de340c39117fd2c347da57c => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:12:22.947894 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a026a4cb-1595-448c-a65c-9f1cf364a5c0 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:12:22.949405 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-a026a4cb-1595-448c-a65c-9f1cf364a5c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:12:22.962316 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a026a4cb-1595-448c-a65c-9f1cf364a5c0 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:12:23.020635 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfa80a5-b1de-4897-bc24-911838bbf116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:12:23.021262 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfa80a5-b1de-4897-bc24-911838bbf116 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:12:23.021539 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfa80a5-b1de-4897-bc24-911838bbf116 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:12:23.021811 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfa80a5-b1de-4897-bc24-911838bbf116 admin admin] PATH_INFO: `/v3/users/fc0ed1f72001432a821862e8902ef70b` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:12:23.022920 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfa80a5-b1de-4897-bc24-911838bbf116 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc0ed1f72001432a821862e8902ef70b)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:12:23.024875 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfa80a5-b1de-4897-bc24-911838bbf116 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:12:23.069706 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-fbfa80a5-b1de-4897-bc24-911838bbf116 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc0ed1f72001432a821862e8902ef70b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:12:23.070340 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-fbfa80a5-b1de-4897-bc24-911838bbf116 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc0ed1f72001432a821862e8902ef70b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:12:23.070942 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-fbfa80a5-b1de-4897-bc24-911838bbf116 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc0ed1f72001432a821862e8902ef70b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:12:23.081461 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:12:23.084146 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-fbfa80a5-b1de-4897-bc24-911838bbf116 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc0ed1f72001432a821862e8902ef70b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:12:23.093036 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-fbfa80a5-b1de-4897-bc24-911838bbf116 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc0ed1f72001432a821862e8902ef70b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:12:23.105992 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1234/2460] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 04:12:22 2022] DELETE /identity/v3/users/fc0ed1f72001432a821862e8902ef70b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:12:23.110531 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa671d6c-3527-44eb-9b66-4c0940b7959f None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:12:23.110992 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa671d6c-3527-44eb-9b66-4c0940b7959f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:12:23.111178 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa671d6c-3527-44eb-9b66-4c0940b7959f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:12:23.131576 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-fa671d6c-3527-44eb-9b66-4c0940b7959f None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:12:23.132880 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-fa671d6c-3527-44eb-9b66-4c0940b7959f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:12:23.190653 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1227/2461] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:12:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:12:23.200851 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfa80a5-b1de-4897-bc24-911838bbf116 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:12:23.202286 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-fbfa80a5-b1de-4897-bc24-911838bbf116 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:12:23.258138 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfa80a5-b1de-4897-bc24-911838bbf116 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:12:23.261749 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33de1e4d-dbd5-4e4e-9388-2c963260b4b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:12:23.262405 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de1e4d-dbd5-4e4e-9388-2c963260b4b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:12:23.262645 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de1e4d-dbd5-4e4e-9388-2c963260b4b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:12:23.262915 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33de1e4d-dbd5-4e4e-9388-2c963260b4b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:12:23.271813 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33de1e4d-dbd5-4e4e-9388-2c963260b4b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:12:23.273260 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33de1e4d-dbd5-4e4e-9388-2c963260b4b1 service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:12:23.282559 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1235/2462] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:12:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:12:23.557598 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa671d6c-3527-44eb-9b66-4c0940b7959f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:12:23.559027 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-fa671d6c-3527-44eb-9b66-4c0940b7959f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:12:23.572174 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa671d6c-3527-44eb-9b66-4c0940b7959f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:12:23.577302 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84749a77-d0a0-42b8-893f-af1d2aeab248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:12:23.578400 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84749a77-d0a0-42b8-893f-af1d2aeab248 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:12:23.578400 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84749a77-d0a0-42b8-893f-af1d2aeab248 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:12:23.578603 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84749a77-d0a0-42b8-893f-af1d2aeab248 admin admin] PATH_INFO: `/v3/projects/c64779465436419fa6e450ddf038e1c6` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:12:23.579593 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84749a77-d0a0-42b8-893f-af1d2aeab248 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c64779465436419fa6e450ddf038e1c6)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:12:23.581655 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84749a77-d0a0-42b8-893f-af1d2aeab248 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:12:23.590612 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:INSERT=2,keystone:DELETE=10,keystone:SELECT=88 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:12:23.630038 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-84749a77-d0a0-42b8-893f-af1d2aeab248 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c64779465436419fa6e450ddf038e1c6', 'request_id': 'req-84749a77-d0a0-42b8-893f-af1d2aeab248'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:12:23.636084 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-84749a77-d0a0-42b8-893f-af1d2aeab248 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c64779465436419fa6e450ddf038e1c6', 'request_id': 'req-84749a77-d0a0-42b8-893f-af1d2aeab248'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:12:23.643233 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-84749a77-d0a0-42b8-893f-af1d2aeab248 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c64779465436419fa6e450ddf038e1c6', 'request_id': 'req-84749a77-d0a0-42b8-893f-af1d2aeab248'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:12:23.653839 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-84749a77-d0a0-42b8-893f-af1d2aeab248 admin admin] The token cache is being invalidate because project c64779465436419fa6e450ddf038e1c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 04:12:23.654124 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-84749a77-d0a0-42b8-893f-af1d2aeab248 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:12:23.654769 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-84749a77-d0a0-42b8-893f-af1d2aeab248 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:12:23.656281 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1228/2463] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 04:12:23 2022] DELETE /identity/v3/projects/c64779465436419fa6e450ddf038e1c6 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 04:12:23.661259 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254eb045-a4f4-4972-b5cf-69a64831c71b None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:12:23.661543 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254eb045-a4f4-4972-b5cf-69a64831c71b None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:12:23.661813 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254eb045-a4f4-4972-b5cf-69a64831c71b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:12:23.681564 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-254eb045-a4f4-4972-b5cf-69a64831c71b None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:12:23.682742 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-254eb045-a4f4-4972-b5cf-69a64831c71b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:12:23.741086 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1236/2464] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:12:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:12:23.749253 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc4dce5-c7ee-4c76-9866-e2eb7797715e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:12:23.749541 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc4dce5-c7ee-4c76-9866-e2eb7797715e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:12:23.749826 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc4dce5-c7ee-4c76-9866-e2eb7797715e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:12:23.769549 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-6dc4dce5-c7ee-4c76-9866-e2eb7797715e None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:12:23.770741 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6dc4dce5-c7ee-4c76-9866-e2eb7797715e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:12:23.779135 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1229/2465] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:12:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:12:23.783181 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254eb045-a4f4-4972-b5cf-69a64831c71b None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:12:23.793043 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254eb045-a4f4-4972-b5cf-69a64831c71b None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:12:23.797508 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f4182d-dfa8-4b81-b2f2-2c967f66842a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:12:23.798094 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f4182d-dfa8-4b81-b2f2-2c967f66842a admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:12:23.798367 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f4182d-dfa8-4b81-b2f2-2c967f66842a admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:12:23.798632 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f4182d-dfa8-4b81-b2f2-2c967f66842a admin admin] PATH_INFO: `/v3/domains` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:12:23.799273 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f4182d-dfa8-4b81-b2f2-2c967f66842a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81504) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 04:12:23.799574 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f4182d-dfa8-4b81-b2f2-2c967f66842a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:12:23.801169 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f4182d-dfa8-4b81-b2f2-2c967f66842a admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:12:23.812543 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1237/2466] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 04:12:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 04:12:23.816694 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc4dce5-c7ee-4c76-9866-e2eb7797715e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:12:23.825626 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc4dce5-c7ee-4c76-9866-e2eb7797715e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:12:23.830212 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71699fe-a1a9-42ae-b5d7-fee610d62242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:12:23.830797 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71699fe-a1a9-42ae-b5d7-fee610d62242 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:12:23.831071 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71699fe-a1a9-42ae-b5d7-fee610d62242 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:12:23.831340 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71699fe-a1a9-42ae-b5d7-fee610d62242 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:12:23.831994 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71699fe-a1a9-42ae-b5d7-fee610d62242 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:12:23.833860 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d71699fe-a1a9-42ae-b5d7-fee610d62242 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:12:23.860614 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1230/2467] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 04:12:23 2022] POST /identity/v3/projects => generated 379 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 04:12:23.864728 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f4182d-dfa8-4b81-b2f2-2c967f66842a admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:12:23.874748 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f4182d-dfa8-4b81-b2f2-2c967f66842a admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:12:23.929687 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23722027-2c13-46c1-9ba0-86d769f0a0f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:12:23.930286 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23722027-2c13-46c1-9ba0-86d769f0a0f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:12:23.930565 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23722027-2c13-46c1-9ba0-86d769f0a0f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:12:23.930838 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23722027-2c13-46c1-9ba0-86d769f0a0f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:12:23.931498 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23722027-2c13-46c1-9ba0-86d769f0a0f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:12:23.933413 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23722027-2c13-46c1-9ba0-86d769f0a0f6 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:12:23.951139 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:12:23.958886 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1238/2468] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 04:12:23 2022] POST /identity/v3/users => generated 290 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:12:23.963395 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71699fe-a1a9-42ae-b5d7-fee610d62242 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:12:23.973530 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d71699fe-a1a9-42ae-b5d7-fee610d62242 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:12:23.978093 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5528add4-5326-46fe-9d71-4c4ec5e94b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:12:23.978686 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5528add4-5326-46fe-9d71-4c4ec5e94b6f admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:12:23.978965 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5528add4-5326-46fe-9d71-4c4ec5e94b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:12:23.979236 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5528add4-5326-46fe-9d71-4c4ec5e94b6f admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:12:23.979871 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5528add4-5326-46fe-9d71-4c4ec5e94b6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:12:23.981473 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5528add4-5326-46fe-9d71-4c4ec5e94b6f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:12:23.991375 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1231/2469] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:12:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:12:23.994906 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23722027-2c13-46c1-9ba0-86d769f0a0f6 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:12:24.005371 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23722027-2c13-46c1-9ba0-86d769f0a0f6 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:12:24.010890 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5571a47-4326-40d5-b378-18b30c60cc23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:12:24.011489 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5571a47-4326-40d5-b378-18b30c60cc23 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:12:24.011763 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5571a47-4326-40d5-b378-18b30c60cc23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:12:24.012065 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5571a47-4326-40d5-b378-18b30c60cc23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:12:24.012700 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5571a47-4326-40d5-b378-18b30c60cc23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:12:24.014304 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5571a47-4326-40d5-b378-18b30c60cc23 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:12:24.024749 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1239/2470] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:12:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:12:24.029237 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5528add4-5326-46fe-9d71-4c4ec5e94b6f admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:12:24.038959 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5528add4-5326-46fe-9d71-4c4ec5e94b6f admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:12:24.043538 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328ce48d-3c8d-46a3-b495-5f165135c7e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:12:24.044210 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ce48d-3c8d-46a3-b495-5f165135c7e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:12:24.044488 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ce48d-3c8d-46a3-b495-5f165135c7e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:12:24.044759 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328ce48d-3c8d-46a3-b495-5f165135c7e2 admin admin] PATH_INFO: `/v3/projects/143ef6c3d11b42c595e2fda02cc5a1b8/users/390f0d31f2ee42d9aba3922cce6e78ee/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:12:24.061516 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328ce48d-3c8d-46a3-b495-5f165135c7e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=143ef6c3d11b42c595e2fda02cc5a1b8, user_id=390f0d31f2ee42d9aba3922cce6e78ee, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:12:24.065260 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-328ce48d-3c8d-46a3-b495-5f165135c7e2 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:12:24.079066 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1232/2471] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 04:12:24 2022] PUT /identity/v3/projects/143ef6c3d11b42c595e2fda02cc5a1b8/users/390f0d31f2ee42d9aba3922cce6e78ee/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:12:24.084033 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb0cd5e-4458-44b4-ac1b-80d2661d6f49 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:12:24.084322 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb0cd5e-4458-44b4-ac1b-80d2661d6f49 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:12:24.084592 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb0cd5e-4458-44b4-ac1b-80d2661d6f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:12:24.103644 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-7bb0cd5e-4458-44b4-ac1b-80d2661d6f49 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:12:24.104926 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-7bb0cd5e-4458-44b4-ac1b-80d2661d6f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:12:24.166176 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1240/2472] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:12:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:12:24.175934 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328ce48d-3c8d-46a3-b495-5f165135c7e2 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:12:24.177810 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-328ce48d-3c8d-46a3-b495-5f165135c7e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:12:24.236331 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328ce48d-3c8d-46a3-b495-5f165135c7e2 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:12:24.239997 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862c6717-4dce-4af7-a78b-243b56c859ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:12:24.240625 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c6717-4dce-4af7-a78b-243b56c859ec service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:12:24.240900 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c6717-4dce-4af7-a78b-243b56c859ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:12:24.241171 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862c6717-4dce-4af7-a78b-243b56c859ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:12:24.250198 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862c6717-4dce-4af7-a78b-243b56c859ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:12:24.251542 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862c6717-4dce-4af7-a78b-243b56c859ec service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:12:24.261584 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1233/2473] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:12:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:12:28.283798 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90038ff5-0208-44ec-8683-457b0e8003b3 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:12:28.284299 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90038ff5-0208-44ec-8683-457b0e8003b3 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:12:28.284408 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90038ff5-0208-44ec-8683-457b0e8003b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:12:28.315571 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-90038ff5-0208-44ec-8683-457b0e8003b3 None None] MFA Rules not processed for user `390f0d31f2ee42d9aba3922cce6e78ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:12:28.316881 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-90038ff5-0208-44ec-8683-457b0e8003b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:12:28.394641 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1241/2474] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:12:28 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:12:28.399075 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862c6717-4dce-4af7-a78b-243b56c859ec service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:12:28.409209 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862c6717-4dce-4af7-a78b-243b56c859ec service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:12:28.413815 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051871e5-a2b3-4a14-8c34-2e8b061a1193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:12:28.414433 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051871e5-a2b3-4a14-8c34-2e8b061a1193 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:12:28.414712 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051871e5-a2b3-4a14-8c34-2e8b061a1193 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:12:28.414984 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051871e5-a2b3-4a14-8c34-2e8b061a1193 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:12:28.415681 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051871e5-a2b3-4a14-8c34-2e8b061a1193 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:12:28.417622 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051871e5-a2b3-4a14-8c34-2e8b061a1193 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:12:28.445630 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1234/2475] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 04:12:28 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:12:28.450186 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90038ff5-0208-44ec-8683-457b0e8003b3 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:12:28.461555 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90038ff5-0208-44ec-8683-457b0e8003b3 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:12:28.538698 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da537df7-d9e5-4986-bb2a-80afb4725c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:12:28.539587 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da537df7-d9e5-4986-bb2a-80afb4725c7c admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:12:28.539864 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da537df7-d9e5-4986-bb2a-80afb4725c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:12:28.540159 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da537df7-d9e5-4986-bb2a-80afb4725c7c admin admin] PATH_INFO: `/v3/users` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:12:28.541329 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da537df7-d9e5-4986-bb2a-80afb4725c7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:12:28.543238 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da537df7-d9e5-4986-bb2a-80afb4725c7c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:12:28.565661 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1242/2476] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 04:12:28 2022] POST /identity/v3/users => generated 295 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 04:12:28.569748 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051871e5-a2b3-4a14-8c34-2e8b061a1193 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:12:28.580417 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051871e5-a2b3-4a14-8c34-2e8b061a1193 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:12:28.585001 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e089c96-129d-4e4a-acdb-ee3e5f79501a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:12:28.585607 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e089c96-129d-4e4a-acdb-ee3e5f79501a admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:12:28.585885 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e089c96-129d-4e4a-acdb-ee3e5f79501a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:12:28.586155 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e089c96-129d-4e4a-acdb-ee3e5f79501a admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:12:28.586791 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e089c96-129d-4e4a-acdb-ee3e5f79501a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:12:28.588455 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e089c96-129d-4e4a-acdb-ee3e5f79501a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:12:28.600560 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1235/2477] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:12:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:12:28.604103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da537df7-d9e5-4986-bb2a-80afb4725c7c admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:12:28.614877 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da537df7-d9e5-4986-bb2a-80afb4725c7c admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:12:28.620445 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6104c8-de8b-46bf-8d89-31d893dff87c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:12:28.621163 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6104c8-de8b-46bf-8d89-31d893dff87c admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:12:28.621439 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6104c8-de8b-46bf-8d89-31d893dff87c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:12:28.621761 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6104c8-de8b-46bf-8d89-31d893dff87c admin admin] PATH_INFO: `/v3/projects/f48a1a56ed8e43f79571a3bbc74045ac/users/1c9e1f35c6a948c593ceb292f7b4e3c4/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:12:28.624465 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:12:28.640171 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6104c8-de8b-46bf-8d89-31d893dff87c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f48a1a56ed8e43f79571a3bbc74045ac, user_id=1c9e1f35c6a948c593ceb292f7b4e3c4, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:12:28.642871 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6104c8-de8b-46bf-8d89-31d893dff87c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:12:28.656512 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1243/2478] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 04:12:28 2022] PUT /identity/v3/projects/f48a1a56ed8e43f79571a3bbc74045ac/users/1c9e1f35c6a948c593ceb292f7b4e3c4/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:12:28.660423 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e089c96-129d-4e4a-acdb-ee3e5f79501a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:12:28.666016 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:12:28.672723 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e089c96-129d-4e4a-acdb-ee3e5f79501a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:12:28.729842 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd67de7-2077-4ebc-98e6-0c5c8a2c3b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:12:28.730379 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd67de7-2077-4ebc-98e6-0c5c8a2c3b92 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:12:28.730675 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd67de7-2077-4ebc-98e6-0c5c8a2c3b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:12:28.730935 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd67de7-2077-4ebc-98e6-0c5c8a2c3b92 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:12:28.731573 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd67de7-2077-4ebc-98e6-0c5c8a2c3b92 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:12:28.733218 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd67de7-2077-4ebc-98e6-0c5c8a2c3b92 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:12:28.742815 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1236/2479] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:12:28 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:12:28.747495 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6104c8-de8b-46bf-8d89-31d893dff87c admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:12:28.757885 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6104c8-de8b-46bf-8d89-31d893dff87c admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:12:28.762483 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be04285-8b7d-4bcb-ade6-29473bb714be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:12:28.763119 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be04285-8b7d-4bcb-ade6-29473bb714be admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:12:28.763399 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be04285-8b7d-4bcb-ade6-29473bb714be admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:12:28.763667 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be04285-8b7d-4bcb-ade6-29473bb714be admin admin] PATH_INFO: `/v3/projects/f48a1a56ed8e43f79571a3bbc74045ac/users/1c9e1f35c6a948c593ceb292f7b4e3c4/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:12:28.765581 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be04285-8b7d-4bcb-ade6-29473bb714be admin admin] RBAC: Authorizing `identity:create_grant(project_id=f48a1a56ed8e43f79571a3bbc74045ac, user_id=1c9e1f35c6a948c593ceb292f7b4e3c4, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:12:28.768231 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be04285-8b7d-4bcb-ade6-29473bb714be admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:12:28.778082 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1244/2480] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 04:12:28 2022] PUT /identity/v3/projects/f48a1a56ed8e43f79571a3bbc74045ac/users/1c9e1f35c6a948c593ceb292f7b4e3c4/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:12:33.383347 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d43054-c16b-4bec-923f-aa4fbdf80bd6 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:12:33.384899 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d43054-c16b-4bec-923f-aa4fbdf80bd6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:12:33.385312 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d43054-c16b-4bec-923f-aa4fbdf80bd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:12:33.422464 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-62d43054-c16b-4bec-923f-aa4fbdf80bd6 None None] MFA Rules not processed for user `1c9e1f35c6a948c593ceb292f7b4e3c4`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:12:33.424076 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-62d43054-c16b-4bec-923f-aa4fbdf80bd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:12:33.528055 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1237/2481] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:12:33 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:12:33.542395 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be04285-8b7d-4bcb-ade6-29473bb714be admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:12:33.544246 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-8be04285-8b7d-4bcb-ade6-29473bb714be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:12:33.641548 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be04285-8b7d-4bcb-ade6-29473bb714be admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:12:33.647633 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e087be6-cb2f-4607-89c2-b007e5a116a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:12:33.648298 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e087be6-cb2f-4607-89c2-b007e5a116a5 service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:12:33.649205 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e087be6-cb2f-4607-89c2-b007e5a116a5 service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:12:33.649487 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e087be6-cb2f-4607-89c2-b007e5a116a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:12:33.717851 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e087be6-cb2f-4607-89c2-b007e5a116a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:12:33.719811 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e087be6-cb2f-4607-89c2-b007e5a116a5 service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:12:33.731220 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1245/2482] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:12:33 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:12:43.520837 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=62 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:12:43.726102 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=68,keystone:INSERT=2 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:12:53.179460 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e76ea5-493f-43ef-af45-de66b66dbcba None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:12:53.180053 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e76ea5-493f-43ef-af45-de66b66dbcba None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:12:53.180183 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e76ea5-493f-43ef-af45-de66b66dbcba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:12:53.208691 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-f9e76ea5-493f-43ef-af45-de66b66dbcba None None] MFA Rules not processed for user `390f0d31f2ee42d9aba3922cce6e78ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:12:53.209817 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f9e76ea5-493f-43ef-af45-de66b66dbcba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:12:53.230008 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1238/2483] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:12:53 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:12:53.274345 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e087be6-cb2f-4607-89c2-b007e5a116a5 service nova] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:12:53.275737 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-8e087be6-cb2f-4607-89c2-b007e5a116a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:12:53.341937 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e087be6-cb2f-4607-89c2-b007e5a116a5 service nova] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:12:53.345469 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347f227c-ae4f-4078-99c2-ebdf68c12bd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:12:53.346068 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f227c-ae4f-4078-99c2-ebdf68c12bd9 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:12:53.346346 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f227c-ae4f-4078-99c2-ebdf68c12bd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:12:53.346618 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347f227c-ae4f-4078-99c2-ebdf68c12bd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:12:53.348332 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-347f227c-ae4f-4078-99c2-ebdf68c12bd9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:12:53.368505 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347f227c-ae4f-4078-99c2-ebdf68c12bd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:12:53.370468 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347f227c-ae4f-4078-99c2-ebdf68c12bd9 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:12:53.381712 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1246/2484] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:12:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:12:53.658710 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e76ea5-493f-43ef-af45-de66b66dbcba None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:12:53.660154 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f9e76ea5-493f-43ef-af45-de66b66dbcba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:12:53.717530 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e76ea5-493f-43ef-af45-de66b66dbcba None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:12:53.721071 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042d1ed3-06c3-4e1b-a7db-515e455d9ef5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:12:53.721673 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042d1ed3-06c3-4e1b-a7db-515e455d9ef5 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:12:53.722006 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042d1ed3-06c3-4e1b-a7db-515e455d9ef5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:12:53.722227 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042d1ed3-06c3-4e1b-a7db-515e455d9ef5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:12:53.724046 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-042d1ed3-06c3-4e1b-a7db-515e455d9ef5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:12:53.735255 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042d1ed3-06c3-4e1b-a7db-515e455d9ef5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:12:53.736706 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042d1ed3-06c3-4e1b-a7db-515e455d9ef5 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:12:53.744569 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1239/2485] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:12:53 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:12:58.913513 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347f227c-ae4f-4078-99c2-ebdf68c12bd9 service cinder] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:12:58.914964 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-347f227c-ae4f-4078-99c2-ebdf68c12bd9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:12:59.003478 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347f227c-ae4f-4078-99c2-ebdf68c12bd9 service cinder] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:12:59.008442 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97f88ef-1484-4ff6-a594-a55cecd36763 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4b9bd741574ddabadd4b7651ebc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:12:59.009041 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f88ef-1484-4ff6-a594-a55cecd36763 service placement] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:12:59.009318 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f88ef-1484-4ff6-a594-a55cecd36763 service placement] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:12:59.010142 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f88ef-1484-4ff6-a594-a55cecd36763 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:12:59.011853 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-c97f88ef-1484-4ff6-a594-a55cecd36763 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:12:59.023022 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97f88ef-1484-4ff6-a594-a55cecd36763 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:12:59.024617 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97f88ef-1484-4ff6-a594-a55cecd36763 service placement] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:12:59.034303 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1247/2486] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:12:58 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:13:03.740979 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=39 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:13:08.475872 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042d1ed3-06c3-4e1b-a7db-515e455d9ef5 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:13:08.494930 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042d1ed3-06c3-4e1b-a7db-515e455d9ef5 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:13:08.498778 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc45962-fbaf-489f-a450-79707e4c2351 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:13:08.499366 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc45962-fbaf-489f-a450-79707e4c2351 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:13:08.499636 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc45962-fbaf-489f-a450-79707e4c2351 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:13:08.499901 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc45962-fbaf-489f-a450-79707e4c2351 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:13:08.508735 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc45962-fbaf-489f-a450-79707e4c2351 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:13:08.510162 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc45962-fbaf-489f-a450-79707e4c2351 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:13:08.519610 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1240/2487] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:13:08 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:13:09.030212 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=74 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:13:17.473023 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97f88ef-1484-4ff6-a594-a55cecd36763 service placement] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:13:17.492488 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97f88ef-1484-4ff6-a594-a55cecd36763 service placement] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:13:17.496761 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea970d1b-3c9f-4b48-b965-333cbf2b9abe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:13:17.497356 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea970d1b-3c9f-4b48-b965-333cbf2b9abe service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:13:17.497888 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea970d1b-3c9f-4b48-b965-333cbf2b9abe service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:13:17.498159 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea970d1b-3c9f-4b48-b965-333cbf2b9abe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:13:17.499869 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ea970d1b-3c9f-4b48-b965-333cbf2b9abe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:13:17.562632 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea970d1b-3c9f-4b48-b965-333cbf2b9abe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:13:17.564114 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea970d1b-3c9f-4b48-b965-333cbf2b9abe service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:13:17.573829 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1248/2488] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:13:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:13:18.515470 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=11 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:13:20.707091 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc45962-fbaf-489f-a450-79707e4c2351 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:13:20.708556 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6fc45962-fbaf-489f-a450-79707e4c2351 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:13:20.723319 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc45962-fbaf-489f-a450-79707e4c2351 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:13:20.727824 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c1952f-0145-474c-acea-8a916734a77d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:13:20.728483 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c1952f-0145-474c-acea-8a916734a77d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:13:20.728789 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c1952f-0145-474c-acea-8a916734a77d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:13:20.729066 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c1952f-0145-474c-acea-8a916734a77d admin admin] PATH_INFO: `/v3/users/7984dc1d452a4c998b44e6d26a299df8` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:13:20.730764 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c1952f-0145-474c-acea-8a916734a77d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7984dc1d452a4c998b44e6d26a299df8)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:13:20.732707 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c1952f-0145-474c-acea-8a916734a77d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:13:20.779854 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d3c1952f-0145-474c-acea-8a916734a77d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7984dc1d452a4c998b44e6d26a299df8'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:13:20.786636 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d3c1952f-0145-474c-acea-8a916734a77d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7984dc1d452a4c998b44e6d26a299df8'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:13:20.787286 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d3c1952f-0145-474c-acea-8a916734a77d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7984dc1d452a4c998b44e6d26a299df8'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:13:20.799997 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d3c1952f-0145-474c-acea-8a916734a77d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7984dc1d452a4c998b44e6d26a299df8'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:13:20.807473 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d3c1952f-0145-474c-acea-8a916734a77d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7984dc1d452a4c998b44e6d26a299df8'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:13:20.813436 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1241/2489] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 04:13:20 2022] DELETE /identity/v3/users/7984dc1d452a4c998b44e6d26a299df8 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:13:20.985796 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea970d1b-3c9f-4b48-b965-333cbf2b9abe service neutron] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:13:20.995995 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea970d1b-3c9f-4b48-b965-333cbf2b9abe service neutron] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:13:21.052854 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cf6f74-d6bd-403c-84b9-3dd56e0da486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:13:21.053407 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf6f74-d6bd-403c-84b9-3dd56e0da486 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:13:21.053682 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf6f74-d6bd-403c-84b9-3dd56e0da486 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:13:21.053949 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf6f74-d6bd-403c-84b9-3dd56e0da486 admin admin] PATH_INFO: `/v3/projects/10a674b48d484716a8a79d518ea0d579` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:13:21.055038 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cf6f74-d6bd-403c-84b9-3dd56e0da486 admin admin] RBAC: Authorizing `identity:delete_project(project_id=10a674b48d484716a8a79d518ea0d579)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:13:21.056956 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cf6f74-d6bd-403c-84b9-3dd56e0da486 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:13:21.102009 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-21cf6f74-d6bd-403c-84b9-3dd56e0da486 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '10a674b48d484716a8a79d518ea0d579', 'request_id': 'req-21cf6f74-d6bd-403c-84b9-3dd56e0da486'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:13:21.107519 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-21cf6f74-d6bd-403c-84b9-3dd56e0da486 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '10a674b48d484716a8a79d518ea0d579', 'request_id': 'req-21cf6f74-d6bd-403c-84b9-3dd56e0da486'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:13:21.113527 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-21cf6f74-d6bd-403c-84b9-3dd56e0da486 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '10a674b48d484716a8a79d518ea0d579', 'request_id': 'req-21cf6f74-d6bd-403c-84b9-3dd56e0da486'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:13:21.125534 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-21cf6f74-d6bd-403c-84b9-3dd56e0da486 admin admin] The token cache is being invalidate because project 10a674b48d484716a8a79d518ea0d579 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 04:13:21.125820 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-21cf6f74-d6bd-403c-84b9-3dd56e0da486 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:13:21.126425 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-21cf6f74-d6bd-403c-84b9-3dd56e0da486 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:13:21.127886 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1249/2490] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 04:13:20 2022] DELETE /identity/v3/projects/10a674b48d484716a8a79d518ea0d579 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:13:21.134336 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc9ff00-d015-468e-a5c4-1a2b12882d21 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:13:21.134629 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc9ff00-d015-468e-a5c4-1a2b12882d21 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:13:21.134906 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc9ff00-d015-468e-a5c4-1a2b12882d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:13:21.155722 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-2dc9ff00-d015-468e-a5c4-1a2b12882d21 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:13:21.156976 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2dc9ff00-d015-468e-a5c4-1a2b12882d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:13:21.215568 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1242/2491] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:13:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 04:13:21.223138 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ddb6b0-93fb-491d-8ca1-cfc615f0fd9f None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:13:21.223426 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ddb6b0-93fb-491d-8ca1-cfc615f0fd9f None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:13:21.223730 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ddb6b0-93fb-491d-8ca1-cfc615f0fd9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:13:21.243048 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-03ddb6b0-93fb-491d-8ca1-cfc615f0fd9f None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:13:21.244277 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-03ddb6b0-93fb-491d-8ca1-cfc615f0fd9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:13:21.252868 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1250/2492] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:13:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:13:21.256926 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc9ff00-d015-468e-a5c4-1a2b12882d21 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:13:21.267112 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc9ff00-d015-468e-a5c4-1a2b12882d21 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:13:21.271788 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b18b8ef-81a6-4016-afb4-c86e0f30f724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:13:21.272417 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b18b8ef-81a6-4016-afb4-c86e0f30f724 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:13:21.272699 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b18b8ef-81a6-4016-afb4-c86e0f30f724 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:13:21.273005 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b18b8ef-81a6-4016-afb4-c86e0f30f724 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:13:21.273646 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b18b8ef-81a6-4016-afb4-c86e0f30f724 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 04:13:21.273951 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b18b8ef-81a6-4016-afb4-c86e0f30f724 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:13:21.275531 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b18b8ef-81a6-4016-afb4-c86e0f30f724 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:13:21.287496 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1243/2493] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 04:13:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 04:13:21.292233 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ddb6b0-93fb-491d-8ca1-cfc615f0fd9f None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:13:21.301729 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ddb6b0-93fb-491d-8ca1-cfc615f0fd9f None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:13:21.306523 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b2e7f6-9b65-42ca-9169-17bb2a4d752f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:13:21.307065 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b2e7f6-9b65-42ca-9169-17bb2a4d752f admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:13:21.307345 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b2e7f6-9b65-42ca-9169-17bb2a4d752f admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:13:21.308489 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b2e7f6-9b65-42ca-9169-17bb2a4d752f admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:13:21.309126 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b2e7f6-9b65-42ca-9169-17bb2a4d752f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:13:21.311047 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b2e7f6-9b65-42ca-9169-17bb2a4d752f admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:13:21.316198 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:13:21.341068 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1251/2494] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 04:13:21 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:13:21.345097 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b18b8ef-81a6-4016-afb4-c86e0f30f724 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:13:21.354975 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b18b8ef-81a6-4016-afb4-c86e0f30f724 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:13:21.410965 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38276383-1ac8-4e6d-943c-65129a109461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:13:21.411577 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38276383-1ac8-4e6d-943c-65129a109461 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:13:21.411858 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38276383-1ac8-4e6d-943c-65129a109461 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:13:21.412163 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38276383-1ac8-4e6d-943c-65129a109461 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:13:21.412804 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38276383-1ac8-4e6d-943c-65129a109461 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:13:21.414773 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38276383-1ac8-4e6d-943c-65129a109461 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:13:21.423343 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:13:21.440325 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1244/2495] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 04:13:21 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 04:13:21.445081 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b2e7f6-9b65-42ca-9169-17bb2a4d752f admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:13:21.462031 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b2e7f6-9b65-42ca-9169-17bb2a4d752f admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:13:21.469286 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e9a819-581c-4a17-bad2-3021b410c32c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:13:21.469731 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e9a819-581c-4a17-bad2-3021b410c32c admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:13:21.470105 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e9a819-581c-4a17-bad2-3021b410c32c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:13:21.472178 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e9a819-581c-4a17-bad2-3021b410c32c admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:13:21.472430 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e9a819-581c-4a17-bad2-3021b410c32c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:13:21.476597 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e9a819-581c-4a17-bad2-3021b410c32c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:13:21.489616 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1252/2496] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:13:21 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:13:21.493721 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38276383-1ac8-4e6d-943c-65129a109461 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:13:21.504701 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38276383-1ac8-4e6d-943c-65129a109461 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:13:21.509392 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e9be84-3c6b-4172-bea4-9d7ae2d415b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:13:21.509998 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e9be84-3c6b-4172-bea4-9d7ae2d415b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:13:21.510282 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e9be84-3c6b-4172-bea4-9d7ae2d415b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:13:21.510564 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e9be84-3c6b-4172-bea4-9d7ae2d415b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:13:21.511237 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e9be84-3c6b-4172-bea4-9d7ae2d415b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:13:21.512856 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e9be84-3c6b-4172-bea4-9d7ae2d415b1 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:13:21.526405 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1245/2497] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:13:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:13:21.531038 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e9a819-581c-4a17-bad2-3021b410c32c admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:13:21.541255 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e9a819-581c-4a17-bad2-3021b410c32c admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:13:21.546735 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d174c48a-3a4a-4961-8471-256bc0889050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:13:21.547393 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d174c48a-3a4a-4961-8471-256bc0889050 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:13:21.547677 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d174c48a-3a4a-4961-8471-256bc0889050 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:13:21.547972 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d174c48a-3a4a-4961-8471-256bc0889050 admin admin] PATH_INFO: `/v3/projects/338cf5433fef44f1a76d460cad6b08f1/users/f518e01f9f8d4a18947fe4cc91e47325/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:13:21.562778 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d174c48a-3a4a-4961-8471-256bc0889050 admin admin] RBAC: Authorizing `identity:create_grant(project_id=338cf5433fef44f1a76d460cad6b08f1, user_id=f518e01f9f8d4a18947fe4cc91e47325, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:13:21.565439 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d174c48a-3a4a-4961-8471-256bc0889050 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:13:21.578669 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1253/2498] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 04:13:21 2022] PUT /identity/v3/projects/338cf5433fef44f1a76d460cad6b08f1/users/f518e01f9f8d4a18947fe4cc91e47325/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 04:13:21.588742 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4230cb-5a72-488f-a974-6f96825ca943 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:13:21.589550 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4230cb-5a72-488f-a974-6f96825ca943 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:13:21.589845 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4230cb-5a72-488f-a974-6f96825ca943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:13:21.617483 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-4f4230cb-5a72-488f-a974-6f96825ca943 None None] MFA Rules not processed for user `f518e01f9f8d4a18947fe4cc91e47325`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:13:21.618689 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4f4230cb-5a72-488f-a974-6f96825ca943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:13:21.683532 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1246/2499] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:13:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:13:21.694717 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d174c48a-3a4a-4961-8471-256bc0889050 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:13:21.696442 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d174c48a-3a4a-4961-8471-256bc0889050 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:13:21.770135 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d174c48a-3a4a-4961-8471-256bc0889050 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:13:21.775633 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ceee1c-fd61-41ed-a9ff-1b6c40466d1a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:13:21.776263 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ceee1c-fd61-41ed-a9ff-1b6c40466d1a service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:13:21.776547 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ceee1c-fd61-41ed-a9ff-1b6c40466d1a service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:13:21.777366 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ceee1c-fd61-41ed-a9ff-1b6c40466d1a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:13:21.784986 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ceee1c-fd61-41ed-a9ff-1b6c40466d1a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:13:21.786584 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ceee1c-fd61-41ed-a9ff-1b6c40466d1a service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:13:21.795244 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1254/2500] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:13:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:13:21.994157 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4230cb-5a72-488f-a974-6f96825ca943 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:13:22.057750 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4230cb-5a72-488f-a974-6f96825ca943 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:13:22.062697 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d41d5f6-43d6-4660-bd34-fb7ad8f295ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:13:22.063305 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41d5f6-43d6-4660-bd34-fb7ad8f295ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:13:22.063609 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41d5f6-43d6-4660-bd34-fb7ad8f295ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:13:22.063880 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d41d5f6-43d6-4660-bd34-fb7ad8f295ac admin admin] PATH_INFO: `/v3/users/f518e01f9f8d4a18947fe4cc91e47325` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:13:22.065054 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d41d5f6-43d6-4660-bd34-fb7ad8f295ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=f518e01f9f8d4a18947fe4cc91e47325)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:13:22.066905 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d41d5f6-43d6-4660-bd34-fb7ad8f295ac admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:13:22.111290 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9d41d5f6-43d6-4660-bd34-fb7ad8f295ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f518e01f9f8d4a18947fe4cc91e47325'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:13:22.118139 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9d41d5f6-43d6-4660-bd34-fb7ad8f295ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f518e01f9f8d4a18947fe4cc91e47325'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:13:22.118785 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9d41d5f6-43d6-4660-bd34-fb7ad8f295ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f518e01f9f8d4a18947fe4cc91e47325'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:13:22.129493 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:INSERT=3,keystone:SELECT=89,keystone:DELETE=8 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:13:22.132050 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9d41d5f6-43d6-4660-bd34-fb7ad8f295ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f518e01f9f8d4a18947fe4cc91e47325'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:13:22.140223 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9d41d5f6-43d6-4660-bd34-fb7ad8f295ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f518e01f9f8d4a18947fe4cc91e47325'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:13:22.145963 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1247/2501] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 04:13:21 2022] DELETE /identity/v3/users/f518e01f9f8d4a18947fe4cc91e47325 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:13:22.150861 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fce43c2-ac2b-47ec-b8dd-a4e6e9db2b69 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:13:22.151399 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fce43c2-ac2b-47ec-b8dd-a4e6e9db2b69 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:13:22.151586 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fce43c2-ac2b-47ec-b8dd-a4e6e9db2b69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:13:22.170298 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-5fce43c2-ac2b-47ec-b8dd-a4e6e9db2b69 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:13:22.171537 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-5fce43c2-ac2b-47ec-b8dd-a4e6e9db2b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:13:22.229052 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1255/2502] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:13:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:13:22.239433 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d41d5f6-43d6-4660-bd34-fb7ad8f295ac admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:13:22.240980 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9d41d5f6-43d6-4660-bd34-fb7ad8f295ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:13:22.297566 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d41d5f6-43d6-4660-bd34-fb7ad8f295ac admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:13:22.301233 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ad6593-b157-43d6-b8b3-d93ab126a0a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:13:22.301834 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad6593-b157-43d6-b8b3-d93ab126a0a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:13:22.302116 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad6593-b157-43d6-b8b3-d93ab126a0a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:13:22.302389 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ad6593-b157-43d6-b8b3-d93ab126a0a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:13:22.312247 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ad6593-b157-43d6-b8b3-d93ab126a0a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:13:22.314143 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ad6593-b157-43d6-b8b3-d93ab126a0a4 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:13:22.323716 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1248/2503] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:13:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:13:22.587182 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fce43c2-ac2b-47ec-b8dd-a4e6e9db2b69 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:13:22.588788 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-5fce43c2-ac2b-47ec-b8dd-a4e6e9db2b69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:13:22.600737 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fce43c2-ac2b-47ec-b8dd-a4e6e9db2b69 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:13:22.605351 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c12fbf9-f80c-495c-be12-204c8dfcd4be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:13:22.606489 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c12fbf9-f80c-495c-be12-204c8dfcd4be admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:13:22.606766 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c12fbf9-f80c-495c-be12-204c8dfcd4be admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:13:22.607042 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c12fbf9-f80c-495c-be12-204c8dfcd4be admin admin] PATH_INFO: `/v3/projects/338cf5433fef44f1a76d460cad6b08f1` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:13:22.608210 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c12fbf9-f80c-495c-be12-204c8dfcd4be admin admin] RBAC: Authorizing `identity:delete_project(project_id=338cf5433fef44f1a76d460cad6b08f1)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:13:22.610103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c12fbf9-f80c-495c-be12-204c8dfcd4be admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:13:22.646651 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:13:22.661562 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5c12fbf9-f80c-495c-be12-204c8dfcd4be admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '338cf5433fef44f1a76d460cad6b08f1', 'request_id': 'req-5c12fbf9-f80c-495c-be12-204c8dfcd4be'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:13:22.667549 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5c12fbf9-f80c-495c-be12-204c8dfcd4be admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '338cf5433fef44f1a76d460cad6b08f1', 'request_id': 'req-5c12fbf9-f80c-495c-be12-204c8dfcd4be'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:13:22.674371 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5c12fbf9-f80c-495c-be12-204c8dfcd4be admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '338cf5433fef44f1a76d460cad6b08f1', 'request_id': 'req-5c12fbf9-f80c-495c-be12-204c8dfcd4be'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:13:22.688167 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5c12fbf9-f80c-495c-be12-204c8dfcd4be admin admin] The token cache is being invalidate because project 338cf5433fef44f1a76d460cad6b08f1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 04:13:22.688457 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5c12fbf9-f80c-495c-be12-204c8dfcd4be admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:13:22.689103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-5c12fbf9-f80c-495c-be12-204c8dfcd4be admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:13:22.690580 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1256/2504] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 04:13:22 2022] DELETE /identity/v3/projects/338cf5433fef44f1a76d460cad6b08f1 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:13:22.695486 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8af514-5b0a-40ae-a805-945df827c2e7 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:13:22.695805 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8af514-5b0a-40ae-a805-945df827c2e7 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:13:22.696118 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8af514-5b0a-40ae-a805-945df827c2e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:13:22.715507 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-7d8af514-5b0a-40ae-a805-945df827c2e7 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:13:22.716721 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7d8af514-5b0a-40ae-a805-945df827c2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:13:22.775649 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1249/2505] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:13:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:13:22.785312 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff2446b-0970-4d45-a3e6-8da331001988 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:13:22.785598 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff2446b-0970-4d45-a3e6-8da331001988 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:13:22.785873 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff2446b-0970-4d45-a3e6-8da331001988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:13:22.805915 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-dff2446b-0970-4d45-a3e6-8da331001988 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:13:22.807095 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-dff2446b-0970-4d45-a3e6-8da331001988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:13:22.814898 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1257/2506] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:13:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:13:22.818867 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8af514-5b0a-40ae-a805-945df827c2e7 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:13:22.828518 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8af514-5b0a-40ae-a805-945df827c2e7 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:13:22.833099 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436e6e0c-c5b8-47be-8e44-ff77ac8166e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:13:22.833639 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e6e0c-c5b8-47be-8e44-ff77ac8166e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:13:22.833916 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e6e0c-c5b8-47be-8e44-ff77ac8166e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:13:22.834189 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436e6e0c-c5b8-47be-8e44-ff77ac8166e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:13:22.834827 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436e6e0c-c5b8-47be-8e44-ff77ac8166e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 04:13:22.835129 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436e6e0c-c5b8-47be-8e44-ff77ac8166e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:13:22.836745 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436e6e0c-c5b8-47be-8e44-ff77ac8166e9 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:13:22.848473 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1250/2507] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 04:13:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 04:13:22.852525 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff2446b-0970-4d45-a3e6-8da331001988 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:13:22.862074 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff2446b-0970-4d45-a3e6-8da331001988 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:13:22.866558 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f72c48-337a-4806-ad38-b79ffe60d961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:13:22.867139 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f72c48-337a-4806-ad38-b79ffe60d961 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:13:22.867412 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f72c48-337a-4806-ad38-b79ffe60d961 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:13:22.867680 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f72c48-337a-4806-ad38-b79ffe60d961 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:13:22.868342 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f72c48-337a-4806-ad38-b79ffe60d961 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:13:22.870201 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f72c48-337a-4806-ad38-b79ffe60d961 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:13:22.896725 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1258/2508] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 04:13:22 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:13:22.900631 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436e6e0c-c5b8-47be-8e44-ff77ac8166e9 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:13:22.911002 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436e6e0c-c5b8-47be-8e44-ff77ac8166e9 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:13:22.969844 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adda78e-96e8-4c0f-820e-200358e45d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:13:22.970441 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adda78e-96e8-4c0f-820e-200358e45d1d admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:13:22.970718 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adda78e-96e8-4c0f-820e-200358e45d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:13:22.970987 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adda78e-96e8-4c0f-820e-200358e45d1d admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:13:22.971616 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adda78e-96e8-4c0f-820e-200358e45d1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:13:22.973598 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adda78e-96e8-4c0f-820e-200358e45d1d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:13:22.993770 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1251/2509] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 04:13:22 2022] POST /identity/v3/users => generated 286 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 04:13:22.998901 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f72c48-337a-4806-ad38-b79ffe60d961 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:13:23.010077 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f72c48-337a-4806-ad38-b79ffe60d961 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:13:23.015542 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d8f5e6-b1a4-4942-9165-7fb009cb5124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:13:23.016221 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d8f5e6-b1a4-4942-9165-7fb009cb5124 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:13:23.016495 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d8f5e6-b1a4-4942-9165-7fb009cb5124 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:13:23.016762 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d8f5e6-b1a4-4942-9165-7fb009cb5124 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:13:23.017421 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d8f5e6-b1a4-4942-9165-7fb009cb5124 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:13:23.019004 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d8f5e6-b1a4-4942-9165-7fb009cb5124 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:13:23.029237 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1259/2510] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:13:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:13:23.033326 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adda78e-96e8-4c0f-820e-200358e45d1d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:13:23.043877 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adda78e-96e8-4c0f-820e-200358e45d1d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:13:23.048511 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fa295c-0ecb-49b7-b3ea-5aa7f3447428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:13:23.049107 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa295c-0ecb-49b7-b3ea-5aa7f3447428 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:13:23.049389 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa295c-0ecb-49b7-b3ea-5aa7f3447428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:13:23.049663 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fa295c-0ecb-49b7-b3ea-5aa7f3447428 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:13:23.050304 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fa295c-0ecb-49b7-b3ea-5aa7f3447428 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:13:23.051886 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fa295c-0ecb-49b7-b3ea-5aa7f3447428 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:13:23.053136 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:13:23.063755 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1252/2511] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:13:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:13:23.068058 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d8f5e6-b1a4-4942-9165-7fb009cb5124 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:13:23.078013 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d8f5e6-b1a4-4942-9165-7fb009cb5124 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:13:23.082945 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ba2673-5deb-404e-afe2-57f4d29e9c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:13:23.083643 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba2673-5deb-404e-afe2-57f4d29e9c0c admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:13:23.083876 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba2673-5deb-404e-afe2-57f4d29e9c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:13:23.084188 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ba2673-5deb-404e-afe2-57f4d29e9c0c admin admin] PATH_INFO: `/v3/projects/29fefed8b0654788bcdd93a126405263/users/bf549d69edf1466c976b6a3b857f5909/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:13:23.099519 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ba2673-5deb-404e-afe2-57f4d29e9c0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=29fefed8b0654788bcdd93a126405263, user_id=bf549d69edf1466c976b6a3b857f5909, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:13:23.102237 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ba2673-5deb-404e-afe2-57f4d29e9c0c admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:13:23.114776 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1260/2512] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 04:13:23 2022] PUT /identity/v3/projects/29fefed8b0654788bcdd93a126405263/users/bf549d69edf1466c976b6a3b857f5909/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:13:23.124822 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77018647-5b3b-4ed7-b066-adc31ed94f6c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:13:23.125109 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77018647-5b3b-4ed7-b066-adc31ed94f6c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:13:23.125379 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77018647-5b3b-4ed7-b066-adc31ed94f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:13:23.152619 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-77018647-5b3b-4ed7-b066-adc31ed94f6c None None] MFA Rules not processed for user `bf549d69edf1466c976b6a3b857f5909`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:13:23.154330 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-77018647-5b3b-4ed7-b066-adc31ed94f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:13:23.217481 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1253/2513] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:13:23 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:13:23.221788 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ba2673-5deb-404e-afe2-57f4d29e9c0c admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:13:23.283341 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ba2673-5deb-404e-afe2-57f4d29e9c0c admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:13:23.287973 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefff3c3-846c-40b0-8c80-8de0265ccea0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:13:23.288583 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefff3c3-846c-40b0-8c80-8de0265ccea0 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:13:23.288859 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefff3c3-846c-40b0-8c80-8de0265ccea0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:13:23.289129 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eefff3c3-846c-40b0-8c80-8de0265ccea0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:13:23.289753 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefff3c3-846c-40b0-8c80-8de0265ccea0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:13:23.291649 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eefff3c3-846c-40b0-8c80-8de0265ccea0 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:13:23.318002 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1261/2514] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 04:13:23 2022] POST /identity/v3/projects => generated 373 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:13:23.322039 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77018647-5b3b-4ed7-b066-adc31ed94f6c None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:13:23.332584 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77018647-5b3b-4ed7-b066-adc31ed94f6c None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:13:23.388621 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e057073-dfdb-493f-83e4-d7ba37ab6af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:13:23.389155 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e057073-dfdb-493f-83e4-d7ba37ab6af3 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:13:23.389433 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e057073-dfdb-493f-83e4-d7ba37ab6af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:13:23.389705 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e057073-dfdb-493f-83e4-d7ba37ab6af3 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:13:23.390346 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e057073-dfdb-493f-83e4-d7ba37ab6af3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:13:23.392267 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e057073-dfdb-493f-83e4-d7ba37ab6af3 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:13:23.413136 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1254/2515] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 04:13:23 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:13:23.417331 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefff3c3-846c-40b0-8c80-8de0265ccea0 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:13:23.428358 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eefff3c3-846c-40b0-8c80-8de0265ccea0 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:13:23.432957 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fd6e81-3b4b-49e9-8063-794ddd2cf921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:13:23.433547 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd6e81-3b4b-49e9-8063-794ddd2cf921 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:13:23.433825 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd6e81-3b4b-49e9-8063-794ddd2cf921 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:13:23.434094 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fd6e81-3b4b-49e9-8063-794ddd2cf921 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:13:23.434725 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fd6e81-3b4b-49e9-8063-794ddd2cf921 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:13:23.436341 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05fd6e81-3b4b-49e9-8063-794ddd2cf921 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:13:23.446665 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1262/2516] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:13:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:13:23.450715 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e057073-dfdb-493f-83e4-d7ba37ab6af3 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:13:23.460791 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e057073-dfdb-493f-83e4-d7ba37ab6af3 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:13:23.465395 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf9c70-b6bd-4f8d-a1d2-f9ddee78d21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:13:23.466578 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf9c70-b6bd-4f8d-a1d2-f9ddee78d21d admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:13:23.466856 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf9c70-b6bd-4f8d-a1d2-f9ddee78d21d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:13:23.467126 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0acf9c70-b6bd-4f8d-a1d2-f9ddee78d21d admin admin] PATH_INFO: `/v3/projects/c082d4b99fca42008b31881cdc399134/users/b184abfa94f242c8980c15dc8bcdf206/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:13:23.483078 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acf9c70-b6bd-4f8d-a1d2-f9ddee78d21d admin admin] RBAC: Authorizing `identity:create_grant(project_id=c082d4b99fca42008b31881cdc399134, user_id=b184abfa94f242c8980c15dc8bcdf206, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:13:23.485899 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0acf9c70-b6bd-4f8d-a1d2-f9ddee78d21d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:13:23.499166 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1255/2517] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 04:13:23 2022] PUT /identity/v3/projects/c082d4b99fca42008b31881cdc399134/users/b184abfa94f242c8980c15dc8bcdf206/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:13:23.503072 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fd6e81-3b4b-49e9-8063-794ddd2cf921 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:13:23.513380 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fd6e81-3b4b-49e9-8063-794ddd2cf921 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:13:23.539538 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:13:23.571411 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a890cc6-2d82-4247-aa82-4261dbe5034b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:13:23.572040 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a890cc6-2d82-4247-aa82-4261dbe5034b admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:13:23.572318 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a890cc6-2d82-4247-aa82-4261dbe5034b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:13:23.572587 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a890cc6-2d82-4247-aa82-4261dbe5034b admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:13:23.573216 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a890cc6-2d82-4247-aa82-4261dbe5034b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:13:23.574784 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a890cc6-2d82-4247-aa82-4261dbe5034b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:13:23.584734 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1263/2518] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:13:23 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:13:23.589233 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf9c70-b6bd-4f8d-a1d2-f9ddee78d21d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:13:23.600282 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0acf9c70-b6bd-4f8d-a1d2-f9ddee78d21d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:13:23.604820 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca60ec7-653a-423a-9e15-63050b1c148b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:13:23.605457 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca60ec7-653a-423a-9e15-63050b1c148b admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:13:23.605733 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca60ec7-653a-423a-9e15-63050b1c148b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:13:23.606000 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca60ec7-653a-423a-9e15-63050b1c148b admin admin] PATH_INFO: `/v3/projects/c082d4b99fca42008b31881cdc399134/users/b184abfa94f242c8980c15dc8bcdf206/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:13:23.607941 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca60ec7-653a-423a-9e15-63050b1c148b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c082d4b99fca42008b31881cdc399134, user_id=b184abfa94f242c8980c15dc8bcdf206, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:13:23.610572 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca60ec7-653a-423a-9e15-63050b1c148b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:13:23.620463 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1256/2519] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 04:13:23 2022] PUT /identity/v3/projects/c082d4b99fca42008b31881cdc399134/users/b184abfa94f242c8980c15dc8bcdf206/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:13:23.630150 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92cf740-73fe-4ad0-8682-51fd47c542fe None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:13:23.630508 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92cf740-73fe-4ad0-8682-51fd47c542fe None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:13:23.630810 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92cf740-73fe-4ad0-8682-51fd47c542fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:13:23.658867 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-b92cf740-73fe-4ad0-8682-51fd47c542fe None None] MFA Rules not processed for user `b184abfa94f242c8980c15dc8bcdf206`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:13:23.660024 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-b92cf740-73fe-4ad0-8682-51fd47c542fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:13:23.725495 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1264/2520] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:13:23 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 04:13:23.737862 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca60ec7-653a-423a-9e15-63050b1c148b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:13:23.739572 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8ca60ec7-653a-423a-9e15-63050b1c148b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:13:23.782096 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:13:23.798200 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca60ec7-653a-423a-9e15-63050b1c148b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:13:23.801857 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3c06f2-4e55-4063-ba2f-e6bff4fdfc38 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:13:23.802455 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c06f2-4e55-4063-ba2f-e6bff4fdfc38 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:13:23.802731 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c06f2-4e55-4063-ba2f-e6bff4fdfc38 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:13:23.803001 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3c06f2-4e55-4063-ba2f-e6bff4fdfc38 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:13:23.811939 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3c06f2-4e55-4063-ba2f-e6bff4fdfc38 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:13:23.813373 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3c06f2-4e55-4063-ba2f-e6bff4fdfc38 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:13:23.822675 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1257/2521] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:13:23 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:13:23.896993 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92cf740-73fe-4ad0-8682-51fd47c542fe None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:13:23.906774 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92cf740-73fe-4ad0-8682-51fd47c542fe None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:13:23.910454 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42666db0-6c7b-49ca-9a00-d92e633e67c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:13:23.910992 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42666db0-6c7b-49ca-9a00-d92e633e67c8 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:13:23.911269 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42666db0-6c7b-49ca-9a00-d92e633e67c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:13:23.911536 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42666db0-6c7b-49ca-9a00-d92e633e67c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:13:23.968011 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42666db0-6c7b-49ca-9a00-d92e633e67c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:13:23.969433 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42666db0-6c7b-49ca-9a00-d92e633e67c8 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:13:23.978192 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1265/2522] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:13:23 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:13:33.818512 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=9 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:13:33.974178 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=78 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:14:05.310114 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3c06f2-4e55-4063-ba2f-e6bff4fdfc38 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:05.371462 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3c06f2-4e55-4063-ba2f-e6bff4fdfc38 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:05.376293 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c70566-461c-4306-aae2-026401ed0c5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:05.376925 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c70566-461c-4306-aae2-026401ed0c5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:05.377205 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c70566-461c-4306-aae2-026401ed0c5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:05.377479 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c70566-461c-4306-aae2-026401ed0c5b admin admin] PATH_INFO: `/v3/users/bf549d69edf1466c976b6a3b857f5909` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:05.378582 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c70566-461c-4306-aae2-026401ed0c5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf549d69edf1466c976b6a3b857f5909)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:05.380520 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c70566-461c-4306-aae2-026401ed0c5b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:05.427336 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-30c70566-461c-4306-aae2-026401ed0c5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf549d69edf1466c976b6a3b857f5909'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:05.434271 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-30c70566-461c-4306-aae2-026401ed0c5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf549d69edf1466c976b6a3b857f5909'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:05.435050 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-30c70566-461c-4306-aae2-026401ed0c5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf549d69edf1466c976b6a3b857f5909'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:05.446724 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-30c70566-461c-4306-aae2-026401ed0c5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf549d69edf1466c976b6a3b857f5909'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:05.454692 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-30c70566-461c-4306-aae2-026401ed0c5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf549d69edf1466c976b6a3b857f5909'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:05.460640 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1258/2523] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 04:14:05 2022] DELETE /identity/v3/users/bf549d69edf1466c976b6a3b857f5909 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:14:05.465094 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f8250-f071-4a43-94f4-07dea975e9f8 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:05.465596 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f8250-f071-4a43-94f4-07dea975e9f8 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:05.465696 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36f8250-f071-4a43-94f4-07dea975e9f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:05.485160 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-d36f8250-f071-4a43-94f4-07dea975e9f8 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:14:05.486340 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d36f8250-f071-4a43-94f4-07dea975e9f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:14:05.544636 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1266/2524] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:14:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:14:05.554232 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c70566-461c-4306-aae2-026401ed0c5b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:05.555619 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-30c70566-461c-4306-aae2-026401ed0c5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:14:05.612649 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c70566-461c-4306-aae2-026401ed0c5b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:05.616266 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed2a439-f6f7-4579-bdf5-605a85ccd246 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:05.616857 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed2a439-f6f7-4579-bdf5-605a85ccd246 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:05.617162 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed2a439-f6f7-4579-bdf5-605a85ccd246 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:05.617432 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed2a439-f6f7-4579-bdf5-605a85ccd246 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:05.626808 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed2a439-f6f7-4579-bdf5-605a85ccd246 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:05.628300 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed2a439-f6f7-4579-bdf5-605a85ccd246 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:05.637524 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1259/2525] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:14:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:14:05.898083 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36f8250-f071-4a43-94f4-07dea975e9f8 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:05.899511 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d36f8250-f071-4a43-94f4-07dea975e9f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:14:05.911500 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36f8250-f071-4a43-94f4-07dea975e9f8 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:05.916318 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b0adfd-78a9-4a7c-9590-a170a9ff718b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:05.916954 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b0adfd-78a9-4a7c-9590-a170a9ff718b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:05.917234 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b0adfd-78a9-4a7c-9590-a170a9ff718b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:05.917506 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b0adfd-78a9-4a7c-9590-a170a9ff718b admin admin] PATH_INFO: `/v3/projects/29fefed8b0654788bcdd93a126405263` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:05.918630 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b0adfd-78a9-4a7c-9590-a170a9ff718b admin admin] RBAC: Authorizing `identity:delete_project(project_id=29fefed8b0654788bcdd93a126405263)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:05.920576 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b0adfd-78a9-4a7c-9590-a170a9ff718b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:05.968712 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-05b0adfd-78a9-4a7c-9590-a170a9ff718b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29fefed8b0654788bcdd93a126405263', 'request_id': 'req-05b0adfd-78a9-4a7c-9590-a170a9ff718b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:05.974100 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-05b0adfd-78a9-4a7c-9590-a170a9ff718b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29fefed8b0654788bcdd93a126405263', 'request_id': 'req-05b0adfd-78a9-4a7c-9590-a170a9ff718b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:05.980164 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-05b0adfd-78a9-4a7c-9590-a170a9ff718b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29fefed8b0654788bcdd93a126405263', 'request_id': 'req-05b0adfd-78a9-4a7c-9590-a170a9ff718b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:05.992113 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-05b0adfd-78a9-4a7c-9590-a170a9ff718b admin admin] The token cache is being invalidate because project 29fefed8b0654788bcdd93a126405263 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 04:14:05.992402 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-05b0adfd-78a9-4a7c-9590-a170a9ff718b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:05.993019 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-05b0adfd-78a9-4a7c-9590-a170a9ff718b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:05.994516 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1267/2526] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 04:14:05 2022] DELETE /identity/v3/projects/29fefed8b0654788bcdd93a126405263 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:14:05.998868 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed2a439-f6f7-4579-bdf5-605a85ccd246 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:06.009184 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed2a439-f6f7-4579-bdf5-605a85ccd246 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:06.027786 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:14:06.071238 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f94dd7-20e2-4dfe-a094-8d666eeb102d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:06.071870 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f94dd7-20e2-4dfe-a094-8d666eeb102d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:06.072178 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f94dd7-20e2-4dfe-a094-8d666eeb102d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:06.072449 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f94dd7-20e2-4dfe-a094-8d666eeb102d admin admin] PATH_INFO: `/v3/users/b184abfa94f242c8980c15dc8bcdf206` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:06.073698 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f94dd7-20e2-4dfe-a094-8d666eeb102d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b184abfa94f242c8980c15dc8bcdf206)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:06.075699 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f94dd7-20e2-4dfe-a094-8d666eeb102d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:06.121781 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-16f94dd7-20e2-4dfe-a094-8d666eeb102d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b184abfa94f242c8980c15dc8bcdf206'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:06.129447 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-16f94dd7-20e2-4dfe-a094-8d666eeb102d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b184abfa94f242c8980c15dc8bcdf206'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:06.130107 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-16f94dd7-20e2-4dfe-a094-8d666eeb102d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b184abfa94f242c8980c15dc8bcdf206'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:06.141829 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-16f94dd7-20e2-4dfe-a094-8d666eeb102d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b184abfa94f242c8980c15dc8bcdf206'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:06.148628 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-16f94dd7-20e2-4dfe-a094-8d666eeb102d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b184abfa94f242c8980c15dc8bcdf206'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:06.154439 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1260/2527] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 04:14:05 2022] DELETE /identity/v3/users/b184abfa94f242c8980c15dc8bcdf206 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:14:06.421479 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b0adfd-78a9-4a7c-9590-a170a9ff718b admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:06.423414 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-05b0adfd-78a9-4a7c-9590-a170a9ff718b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:14:06.436975 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b0adfd-78a9-4a7c-9590-a170a9ff718b admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:06.494960 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cdd54ac-5b4f-4f3f-b0e1-726ed9be4d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:06.495579 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd54ac-5b4f-4f3f-b0e1-726ed9be4d25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:06.495842 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd54ac-5b4f-4f3f-b0e1-726ed9be4d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:06.496134 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cdd54ac-5b4f-4f3f-b0e1-726ed9be4d25 admin admin] PATH_INFO: `/v3/projects/c082d4b99fca42008b31881cdc399134` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:06.497257 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdd54ac-5b4f-4f3f-b0e1-726ed9be4d25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c082d4b99fca42008b31881cdc399134)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:06.499423 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cdd54ac-5b4f-4f3f-b0e1-726ed9be4d25 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:06.528774 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:14:06.547009 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-0cdd54ac-5b4f-4f3f-b0e1-726ed9be4d25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c082d4b99fca42008b31881cdc399134', 'request_id': 'req-0cdd54ac-5b4f-4f3f-b0e1-726ed9be4d25'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:06.553266 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-0cdd54ac-5b4f-4f3f-b0e1-726ed9be4d25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c082d4b99fca42008b31881cdc399134', 'request_id': 'req-0cdd54ac-5b4f-4f3f-b0e1-726ed9be4d25'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:06.559456 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-0cdd54ac-5b4f-4f3f-b0e1-726ed9be4d25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c082d4b99fca42008b31881cdc399134', 'request_id': 'req-0cdd54ac-5b4f-4f3f-b0e1-726ed9be4d25'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:06.571671 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-0cdd54ac-5b4f-4f3f-b0e1-726ed9be4d25 admin admin] The token cache is being invalidate because project c082d4b99fca42008b31881cdc399134 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 04:14:06.571977 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-0cdd54ac-5b4f-4f3f-b0e1-726ed9be4d25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:06.572599 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-0cdd54ac-5b4f-4f3f-b0e1-726ed9be4d25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:06.574101 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1268/2528] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 04:14:06 2022] DELETE /identity/v3/projects/c082d4b99fca42008b31881cdc399134 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:14:06.579403 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96fd43f-a4fe-4e2c-a755-e770dbb05513 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:06.579893 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96fd43f-a4fe-4e2c-a755-e770dbb05513 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:06.580036 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96fd43f-a4fe-4e2c-a755-e770dbb05513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:06.599713 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-b96fd43f-a4fe-4e2c-a755-e770dbb05513 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:14:06.600940 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b96fd43f-a4fe-4e2c-a755-e770dbb05513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:14:06.658947 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1261/2529] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:14:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:14:06.666467 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620bb385-34d7-43ee-9181-4dc3c622d3aa None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:06.666844 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620bb385-34d7-43ee-9181-4dc3c622d3aa None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:06.667144 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620bb385-34d7-43ee-9181-4dc3c622d3aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:06.687341 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-620bb385-34d7-43ee-9181-4dc3c622d3aa None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:14:06.689213 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-620bb385-34d7-43ee-9181-4dc3c622d3aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:14:06.696942 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1269/2530] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:14:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:14:06.701162 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96fd43f-a4fe-4e2c-a755-e770dbb05513 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:06.711476 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96fd43f-a4fe-4e2c-a755-e770dbb05513 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:06.716120 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b258fa-948e-41fd-9003-a7a180a17a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:06.716719 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b258fa-948e-41fd-9003-a7a180a17a34 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:06.717001 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b258fa-948e-41fd-9003-a7a180a17a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:06.717277 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b258fa-948e-41fd-9003-a7a180a17a34 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:06.717923 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b258fa-948e-41fd-9003-a7a180a17a34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 04:14:06.718228 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b258fa-948e-41fd-9003-a7a180a17a34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:06.719826 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b258fa-948e-41fd-9003-a7a180a17a34 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:06.731186 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1262/2531] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 04:14:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 04:14:06.735174 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620bb385-34d7-43ee-9181-4dc3c622d3aa None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:06.744334 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620bb385-34d7-43ee-9181-4dc3c622d3aa None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:06.749114 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a734217-0193-462f-8c22-b2d0f1dcd9df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:06.749707 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a734217-0193-462f-8c22-b2d0f1dcd9df admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:06.749981 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a734217-0193-462f-8c22-b2d0f1dcd9df admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:06.750249 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a734217-0193-462f-8c22-b2d0f1dcd9df admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:06.750869 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a734217-0193-462f-8c22-b2d0f1dcd9df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:06.753299 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a734217-0193-462f-8c22-b2d0f1dcd9df admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:06.780014 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1270/2532] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 04:14:06 2022] POST /identity/v3/projects => generated 365 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 04:14:06.783873 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b258fa-948e-41fd-9003-a7a180a17a34 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:06.793814 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b258fa-948e-41fd-9003-a7a180a17a34 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:06.820113 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:14:06.852178 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1928aa10-5079-4239-84ae-ad95146e541e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:06.852779 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1928aa10-5079-4239-84ae-ad95146e541e admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:06.853055 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1928aa10-5079-4239-84ae-ad95146e541e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:06.853323 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1928aa10-5079-4239-84ae-ad95146e541e admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:06.853949 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1928aa10-5079-4239-84ae-ad95146e541e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:06.855840 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1928aa10-5079-4239-84ae-ad95146e541e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:06.877875 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1263/2533] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 04:14:06 2022] POST /identity/v3/users => generated 283 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:14:06.882074 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a734217-0193-462f-8c22-b2d0f1dcd9df admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:06.892275 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a734217-0193-462f-8c22-b2d0f1dcd9df admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:06.896816 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e98af5-7483-488a-900d-05fa2cfd0282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:06.897411 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e98af5-7483-488a-900d-05fa2cfd0282 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:06.897692 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e98af5-7483-488a-900d-05fa2cfd0282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:06.897964 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e98af5-7483-488a-900d-05fa2cfd0282 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:06.898602 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e98af5-7483-488a-900d-05fa2cfd0282 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:06.900239 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e98af5-7483-488a-900d-05fa2cfd0282 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:06.909935 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1271/2534] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:14:06 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:14:06.913753 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1928aa10-5079-4239-84ae-ad95146e541e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:06.924452 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1928aa10-5079-4239-84ae-ad95146e541e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:06.928991 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54771d53-27d5-48d1-a87a-3217b23957ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:06.929581 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54771d53-27d5-48d1-a87a-3217b23957ab admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:06.929857 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54771d53-27d5-48d1-a87a-3217b23957ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:06.930128 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54771d53-27d5-48d1-a87a-3217b23957ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:06.930786 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54771d53-27d5-48d1-a87a-3217b23957ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:06.932407 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54771d53-27d5-48d1-a87a-3217b23957ab admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:06.943993 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1264/2535] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:14:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:14:06.947709 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e98af5-7483-488a-900d-05fa2cfd0282 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:06.957585 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e98af5-7483-488a-900d-05fa2cfd0282 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:06.962151 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae905a5-3dec-44ef-b72e-c5d0af97d74d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:06.962786 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae905a5-3dec-44ef-b72e-c5d0af97d74d admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:06.963064 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae905a5-3dec-44ef-b72e-c5d0af97d74d admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:06.963336 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae905a5-3dec-44ef-b72e-c5d0af97d74d admin admin] PATH_INFO: `/v3/projects/d9b08a1467d04ce2b45c94cd7746b3b8/users/2cf21aa00bcf421a857aba97b5dae692/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:06.977538 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae905a5-3dec-44ef-b72e-c5d0af97d74d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9b08a1467d04ce2b45c94cd7746b3b8, user_id=2cf21aa00bcf421a857aba97b5dae692, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:06.980202 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae905a5-3dec-44ef-b72e-c5d0af97d74d admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:06.993215 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1272/2536] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 04:14:06 2022] PUT /identity/v3/projects/d9b08a1467d04ce2b45c94cd7746b3b8/users/2cf21aa00bcf421a857aba97b5dae692/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:14:07.005259 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403373d4-e66e-4305-a0cb-5a0d3ac95887 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:07.005552 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403373d4-e66e-4305-a0cb-5a0d3ac95887 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:07.005832 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403373d4-e66e-4305-a0cb-5a0d3ac95887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:07.033004 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-403373d4-e66e-4305-a0cb-5a0d3ac95887 None None] MFA Rules not processed for user `2cf21aa00bcf421a857aba97b5dae692`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:14:07.034196 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-403373d4-e66e-4305-a0cb-5a0d3ac95887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:14:07.097532 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1265/2537] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:14:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:14:07.102094 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae905a5-3dec-44ef-b72e-c5d0af97d74d admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:07.162311 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae905a5-3dec-44ef-b72e-c5d0af97d74d admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:07.167207 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eb8bfa-ab60-49ab-acb2-6e7b085d2e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:07.168053 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eb8bfa-ab60-49ab-acb2-6e7b085d2e81 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:07.168454 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eb8bfa-ab60-49ab-acb2-6e7b085d2e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:07.168879 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eb8bfa-ab60-49ab-acb2-6e7b085d2e81 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:07.169589 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eb8bfa-ab60-49ab-acb2-6e7b085d2e81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:07.171870 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38eb8bfa-ab60-49ab-acb2-6e7b085d2e81 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:07.198753 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1273/2538] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 04:14:07 2022] POST /identity/v3/projects => generated 365 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:14:07.202773 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403373d4-e66e-4305-a0cb-5a0d3ac95887 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:07.213548 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403373d4-e66e-4305-a0cb-5a0d3ac95887 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:07.269052 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aee5d12-c0e2-4ade-82d9-1c4c7ca39e1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:07.269651 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee5d12-c0e2-4ade-82d9-1c4c7ca39e1d admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:07.269928 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee5d12-c0e2-4ade-82d9-1c4c7ca39e1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:07.270251 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee5d12-c0e2-4ade-82d9-1c4c7ca39e1d admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:07.270891 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aee5d12-c0e2-4ade-82d9-1c4c7ca39e1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:07.272817 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aee5d12-c0e2-4ade-82d9-1c4c7ca39e1d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:07.294022 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1266/2539] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 04:14:07 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:14:07.298457 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eb8bfa-ab60-49ab-acb2-6e7b085d2e81 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:07.309049 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eb8bfa-ab60-49ab-acb2-6e7b085d2e81 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:07.313658 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fed2cb-a13f-48da-8d12-d737f72a1d03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:07.314248 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fed2cb-a13f-48da-8d12-d737f72a1d03 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:07.314527 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fed2cb-a13f-48da-8d12-d737f72a1d03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:07.314795 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fed2cb-a13f-48da-8d12-d737f72a1d03 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:07.315468 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fed2cb-a13f-48da-8d12-d737f72a1d03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:07.317234 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fed2cb-a13f-48da-8d12-d737f72a1d03 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:07.327592 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1274/2540] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:14:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:14:07.331593 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aee5d12-c0e2-4ade-82d9-1c4c7ca39e1d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:07.341934 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aee5d12-c0e2-4ade-82d9-1c4c7ca39e1d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:07.347216 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705cf33e-c228-420a-aed9-52dd9837af18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:07.347857 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705cf33e-c228-420a-aed9-52dd9837af18 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:07.348172 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705cf33e-c228-420a-aed9-52dd9837af18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:07.348446 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705cf33e-c228-420a-aed9-52dd9837af18 admin admin] PATH_INFO: `/v3/projects/e89ca93f2d2d4908b765a59d05665545/users/2458f2c6fc7643f7ae152f1f878d097a/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:07.363924 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705cf33e-c228-420a-aed9-52dd9837af18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e89ca93f2d2d4908b765a59d05665545, user_id=2458f2c6fc7643f7ae152f1f878d097a, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:07.366569 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705cf33e-c228-420a-aed9-52dd9837af18 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:07.380284 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1267/2541] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 04:14:07 2022] PUT /identity/v3/projects/e89ca93f2d2d4908b765a59d05665545/users/2458f2c6fc7643f7ae152f1f878d097a/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 04:14:07.384268 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fed2cb-a13f-48da-8d12-d737f72a1d03 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:07.394129 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fed2cb-a13f-48da-8d12-d737f72a1d03 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:07.414307 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:14:07.452853 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839d5da3-68da-47b6-8b1e-a626947f0ad2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:07.453465 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d5da3-68da-47b6-8b1e-a626947f0ad2 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:07.453745 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d5da3-68da-47b6-8b1e-a626947f0ad2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:07.454042 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839d5da3-68da-47b6-8b1e-a626947f0ad2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:07.454704 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839d5da3-68da-47b6-8b1e-a626947f0ad2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:07.456373 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839d5da3-68da-47b6-8b1e-a626947f0ad2 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:07.465975 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1275/2542] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:14:07 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:14:07.470553 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705cf33e-c228-420a-aed9-52dd9837af18 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:07.476095 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:14:07.481931 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705cf33e-c228-420a-aed9-52dd9837af18 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:07.487502 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfecc0e-c2de-4faf-87b9-8f8c0fba0341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:07.488177 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfecc0e-c2de-4faf-87b9-8f8c0fba0341 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:07.488461 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfecc0e-c2de-4faf-87b9-8f8c0fba0341 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:07.488733 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfecc0e-c2de-4faf-87b9-8f8c0fba0341 admin admin] PATH_INFO: `/v3/projects/e89ca93f2d2d4908b765a59d05665545/users/2458f2c6fc7643f7ae152f1f878d097a/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:07.490615 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfecc0e-c2de-4faf-87b9-8f8c0fba0341 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e89ca93f2d2d4908b765a59d05665545, user_id=2458f2c6fc7643f7ae152f1f878d097a, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:07.493328 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfecc0e-c2de-4faf-87b9-8f8c0fba0341 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:07.503618 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1268/2543] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 04:14:07 2022] PUT /identity/v3/projects/e89ca93f2d2d4908b765a59d05665545/users/2458f2c6fc7643f7ae152f1f878d097a/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:14:07.513919 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acdfefd-8834-45d7-a803-e488eb17e826 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:07.514395 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acdfefd-8834-45d7-a803-e488eb17e826 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:07.514503 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acdfefd-8834-45d7-a803-e488eb17e826 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:07.543460 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-2acdfefd-8834-45d7-a803-e488eb17e826 None None] MFA Rules not processed for user `2458f2c6fc7643f7ae152f1f878d097a`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:14:07.544752 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-2acdfefd-8834-45d7-a803-e488eb17e826 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:14:07.611678 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1276/2544] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:14:07 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:14:07.623244 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfecc0e-c2de-4faf-87b9-8f8c0fba0341 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:07.625025 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1cfecc0e-c2de-4faf-87b9-8f8c0fba0341 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:14:07.681830 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfecc0e-c2de-4faf-87b9-8f8c0fba0341 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:07.685479 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9395543a-b072-4f67-a007-5bb72f9263dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:07.686077 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9395543a-b072-4f67-a007-5bb72f9263dc service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:07.686358 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9395543a-b072-4f67-a007-5bb72f9263dc service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:07.686633 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9395543a-b072-4f67-a007-5bb72f9263dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:07.695484 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9395543a-b072-4f67-a007-5bb72f9263dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:07.696937 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9395543a-b072-4f67-a007-5bb72f9263dc service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:07.706791 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1269/2545] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:14:07 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:14:07.781094 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acdfefd-8834-45d7-a803-e488eb17e826 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:07.791247 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acdfefd-8834-45d7-a803-e488eb17e826 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:07.794849 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d19516e-0900-4d3b-9767-9d5ac9f4b7c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:07.795390 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d19516e-0900-4d3b-9767-9d5ac9f4b7c0 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:07.795666 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d19516e-0900-4d3b-9767-9d5ac9f4b7c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:07.795935 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d19516e-0900-4d3b-9767-9d5ac9f4b7c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:07.852172 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d19516e-0900-4d3b-9767-9d5ac9f4b7c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:07.853604 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d19516e-0900-4d3b-9767-9d5ac9f4b7c0 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:07.862496 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1277/2546] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:14:07 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:14:17.702070 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:14:17.858475 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=80 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:14:18.051857 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9395543a-b072-4f67-a007-5bb72f9263dc service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:18.113327 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9395543a-b072-4f67-a007-5bb72f9263dc service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:18.117751 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fb4ec2-bb98-4aad-b1cc-ed795864117e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:18.118363 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fb4ec2-bb98-4aad-b1cc-ed795864117e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:18.118639 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fb4ec2-bb98-4aad-b1cc-ed795864117e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:18.118909 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fb4ec2-bb98-4aad-b1cc-ed795864117e admin admin] PATH_INFO: `/v3/users/2cf21aa00bcf421a857aba97b5dae692` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:18.120027 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fb4ec2-bb98-4aad-b1cc-ed795864117e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2cf21aa00bcf421a857aba97b5dae692)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:18.121919 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fb4ec2-bb98-4aad-b1cc-ed795864117e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:18.166594 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-97fb4ec2-bb98-4aad-b1cc-ed795864117e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2cf21aa00bcf421a857aba97b5dae692'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:18.173885 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-97fb4ec2-bb98-4aad-b1cc-ed795864117e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2cf21aa00bcf421a857aba97b5dae692'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:18.174720 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-97fb4ec2-bb98-4aad-b1cc-ed795864117e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2cf21aa00bcf421a857aba97b5dae692'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:18.186353 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-97fb4ec2-bb98-4aad-b1cc-ed795864117e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2cf21aa00bcf421a857aba97b5dae692'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:18.193901 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-97fb4ec2-bb98-4aad-b1cc-ed795864117e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2cf21aa00bcf421a857aba97b5dae692'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:18.199570 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1270/2547] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 04:14:18 2022] DELETE /identity/v3/users/2cf21aa00bcf421a857aba97b5dae692 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:14:18.204310 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d4cfe-f40c-408c-b98e-a7db5cd36ad8 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:18.204608 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d4cfe-f40c-408c-b98e-a7db5cd36ad8 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:18.204888 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d4cfe-f40c-408c-b98e-a7db5cd36ad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:18.224851 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-5f0d4cfe-f40c-408c-b98e-a7db5cd36ad8 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:14:18.226036 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-5f0d4cfe-f40c-408c-b98e-a7db5cd36ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:14:18.284064 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1278/2548] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:14:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:14:18.293481 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fb4ec2-bb98-4aad-b1cc-ed795864117e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:18.294847 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-97fb4ec2-bb98-4aad-b1cc-ed795864117e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:14:18.351409 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fb4ec2-bb98-4aad-b1cc-ed795864117e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:18.355251 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a0fdd4-00e5-46e8-8694-3058b359cad9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:18.355846 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a0fdd4-00e5-46e8-8694-3058b359cad9 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:18.356153 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a0fdd4-00e5-46e8-8694-3058b359cad9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:18.356423 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a0fdd4-00e5-46e8-8694-3058b359cad9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:18.365788 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a0fdd4-00e5-46e8-8694-3058b359cad9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:18.367127 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a0fdd4-00e5-46e8-8694-3058b359cad9 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:18.377666 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1271/2549] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:14:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:14:18.648322 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0d4cfe-f40c-408c-b98e-a7db5cd36ad8 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:18.649746 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-5f0d4cfe-f40c-408c-b98e-a7db5cd36ad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:14:18.662512 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0d4cfe-f40c-408c-b98e-a7db5cd36ad8 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:18.667146 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350d8cce-4a77-42d0-8886-278fe5ffb44b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:18.667783 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350d8cce-4a77-42d0-8886-278fe5ffb44b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:18.668099 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350d8cce-4a77-42d0-8886-278fe5ffb44b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:18.668378 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350d8cce-4a77-42d0-8886-278fe5ffb44b admin admin] PATH_INFO: `/v3/projects/d9b08a1467d04ce2b45c94cd7746b3b8` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:18.669506 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350d8cce-4a77-42d0-8886-278fe5ffb44b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9b08a1467d04ce2b45c94cd7746b3b8)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:18.671418 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350d8cce-4a77-42d0-8886-278fe5ffb44b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:18.717533 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-350d8cce-4a77-42d0-8886-278fe5ffb44b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9b08a1467d04ce2b45c94cd7746b3b8', 'request_id': 'req-350d8cce-4a77-42d0-8886-278fe5ffb44b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:18.723617 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-350d8cce-4a77-42d0-8886-278fe5ffb44b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9b08a1467d04ce2b45c94cd7746b3b8', 'request_id': 'req-350d8cce-4a77-42d0-8886-278fe5ffb44b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:18.730074 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-350d8cce-4a77-42d0-8886-278fe5ffb44b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9b08a1467d04ce2b45c94cd7746b3b8', 'request_id': 'req-350d8cce-4a77-42d0-8886-278fe5ffb44b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:18.742207 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-350d8cce-4a77-42d0-8886-278fe5ffb44b admin admin] The token cache is being invalidate because project d9b08a1467d04ce2b45c94cd7746b3b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 04:14:18.742489 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-350d8cce-4a77-42d0-8886-278fe5ffb44b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:18.743121 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-350d8cce-4a77-42d0-8886-278fe5ffb44b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:18.744618 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1279/2550] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 04:14:18 2022] DELETE /identity/v3/projects/d9b08a1467d04ce2b45c94cd7746b3b8 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:14:18.748796 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a0fdd4-00e5-46e8-8694-3058b359cad9 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:18.758234 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a0fdd4-00e5-46e8-8694-3058b359cad9 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:18.775691 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:14:18.815617 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e878757a-39f9-4560-bab9-daf1e67fb294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:18.816283 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e878757a-39f9-4560-bab9-daf1e67fb294 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:18.816588 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e878757a-39f9-4560-bab9-daf1e67fb294 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:18.816861 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e878757a-39f9-4560-bab9-daf1e67fb294 admin admin] PATH_INFO: `/v3/users/2458f2c6fc7643f7ae152f1f878d097a` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:18.817981 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e878757a-39f9-4560-bab9-daf1e67fb294 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2458f2c6fc7643f7ae152f1f878d097a)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:18.819966 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e878757a-39f9-4560-bab9-daf1e67fb294 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:18.865667 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e878757a-39f9-4560-bab9-daf1e67fb294 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2458f2c6fc7643f7ae152f1f878d097a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:18.872884 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e878757a-39f9-4560-bab9-daf1e67fb294 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2458f2c6fc7643f7ae152f1f878d097a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:18.873553 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e878757a-39f9-4560-bab9-daf1e67fb294 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2458f2c6fc7643f7ae152f1f878d097a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:18.885261 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e878757a-39f9-4560-bab9-daf1e67fb294 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2458f2c6fc7643f7ae152f1f878d097a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:18.892865 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e878757a-39f9-4560-bab9-daf1e67fb294 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2458f2c6fc7643f7ae152f1f878d097a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:18.898650 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1272/2551] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 04:14:18 2022] DELETE /identity/v3/users/2458f2c6fc7643f7ae152f1f878d097a => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:14:19.164356 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350d8cce-4a77-42d0-8886-278fe5ffb44b admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:19.165943 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-350d8cce-4a77-42d0-8886-278fe5ffb44b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:14:19.180774 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350d8cce-4a77-42d0-8886-278fe5ffb44b admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:19.241719 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b30b994-c6db-4995-a07a-0ab81e29ca0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:19.242356 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b30b994-c6db-4995-a07a-0ab81e29ca0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:19.242639 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b30b994-c6db-4995-a07a-0ab81e29ca0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:19.242912 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b30b994-c6db-4995-a07a-0ab81e29ca0b admin admin] PATH_INFO: `/v3/projects/e89ca93f2d2d4908b765a59d05665545` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:19.244135 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b30b994-c6db-4995-a07a-0ab81e29ca0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=e89ca93f2d2d4908b765a59d05665545)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:19.246027 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b30b994-c6db-4995-a07a-0ab81e29ca0b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:19.276829 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:14:19.294870 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-4b30b994-c6db-4995-a07a-0ab81e29ca0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e89ca93f2d2d4908b765a59d05665545', 'request_id': 'req-4b30b994-c6db-4995-a07a-0ab81e29ca0b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:19.300464 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-4b30b994-c6db-4995-a07a-0ab81e29ca0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e89ca93f2d2d4908b765a59d05665545', 'request_id': 'req-4b30b994-c6db-4995-a07a-0ab81e29ca0b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:19.306740 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-4b30b994-c6db-4995-a07a-0ab81e29ca0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e89ca93f2d2d4908b765a59d05665545', 'request_id': 'req-4b30b994-c6db-4995-a07a-0ab81e29ca0b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:19.319634 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-4b30b994-c6db-4995-a07a-0ab81e29ca0b admin admin] The token cache is being invalidate because project e89ca93f2d2d4908b765a59d05665545 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 04:14:19.319922 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-4b30b994-c6db-4995-a07a-0ab81e29ca0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:19.320556 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-4b30b994-c6db-4995-a07a-0ab81e29ca0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:19.322072 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1280/2552] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 04:14:19 2022] DELETE /identity/v3/projects/e89ca93f2d2d4908b765a59d05665545 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:14:19.327001 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea0c650-7f9b-41d9-889f-de63ce6be224 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:19.327289 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea0c650-7f9b-41d9-889f-de63ce6be224 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:19.327564 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea0c650-7f9b-41d9-889f-de63ce6be224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:19.347714 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-7ea0c650-7f9b-41d9-889f-de63ce6be224 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:14:19.348958 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7ea0c650-7f9b-41d9-889f-de63ce6be224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:14:19.406863 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1273/2553] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:14:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:14:19.414536 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a49b8b-3dc3-4201-a2be-ccdfab2e47a7 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:19.414825 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a49b8b-3dc3-4201-a2be-ccdfab2e47a7 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:19.415102 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a49b8b-3dc3-4201-a2be-ccdfab2e47a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:19.434474 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-e8a49b8b-3dc3-4201-a2be-ccdfab2e47a7 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:14:19.435669 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-e8a49b8b-3dc3-4201-a2be-ccdfab2e47a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:14:19.444292 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1281/2554] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:14:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:14:19.448326 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea0c650-7f9b-41d9-889f-de63ce6be224 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:19.458702 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea0c650-7f9b-41d9-889f-de63ce6be224 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:19.463306 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab46be93-4b40-468e-aa14-99c113119517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:19.463926 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab46be93-4b40-468e-aa14-99c113119517 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:19.464239 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab46be93-4b40-468e-aa14-99c113119517 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:19.464515 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab46be93-4b40-468e-aa14-99c113119517 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:19.465178 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab46be93-4b40-468e-aa14-99c113119517 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 04:14:19.465488 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab46be93-4b40-468e-aa14-99c113119517 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:19.467075 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab46be93-4b40-468e-aa14-99c113119517 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:19.478897 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1274/2555] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 04:14:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 04:14:19.483103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a49b8b-3dc3-4201-a2be-ccdfab2e47a7 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:19.492610 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a49b8b-3dc3-4201-a2be-ccdfab2e47a7 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:19.497572 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2528cee-1e7a-4410-9840-6a47fe3ea998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:19.498165 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2528cee-1e7a-4410-9840-6a47fe3ea998 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:19.498439 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2528cee-1e7a-4410-9840-6a47fe3ea998 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:19.498706 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2528cee-1e7a-4410-9840-6a47fe3ea998 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:19.499338 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2528cee-1e7a-4410-9840-6a47fe3ea998 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:19.501247 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2528cee-1e7a-4410-9840-6a47fe3ea998 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:19.529105 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1282/2556] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 04:14:19 2022] POST /identity/v3/projects => generated 391 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:14:19.533203 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab46be93-4b40-468e-aa14-99c113119517 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:19.543401 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab46be93-4b40-468e-aa14-99c113119517 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:19.569166 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:14:19.601552 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40d9dd5-d71f-4db1-a8fb-ef72cd066d4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:19.602178 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40d9dd5-d71f-4db1-a8fb-ef72cd066d4f admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:19.602441 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40d9dd5-d71f-4db1-a8fb-ef72cd066d4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:19.602714 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40d9dd5-d71f-4db1-a8fb-ef72cd066d4f admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:19.603382 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40d9dd5-d71f-4db1-a8fb-ef72cd066d4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:19.605336 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40d9dd5-d71f-4db1-a8fb-ef72cd066d4f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:19.625687 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1275/2557] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 04:14:19 2022] POST /identity/v3/users => generated 296 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 04:14:19.630475 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2528cee-1e7a-4410-9840-6a47fe3ea998 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:19.640778 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2528cee-1e7a-4410-9840-6a47fe3ea998 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:19.645465 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370ec60e-58ef-4d88-be78-92f2d4bc52bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:19.646065 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ec60e-58ef-4d88-be78-92f2d4bc52bb admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:19.646347 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ec60e-58ef-4d88-be78-92f2d4bc52bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:19.646623 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370ec60e-58ef-4d88-be78-92f2d4bc52bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:19.647296 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370ec60e-58ef-4d88-be78-92f2d4bc52bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:19.648924 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370ec60e-58ef-4d88-be78-92f2d4bc52bb admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:19.659043 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1283/2558] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:14:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:14:19.662750 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40d9dd5-d71f-4db1-a8fb-ef72cd066d4f admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:19.672624 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40d9dd5-d71f-4db1-a8fb-ef72cd066d4f admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:19.677792 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d386b666-b247-4f7e-be58-b2032d04c662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:19.678379 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d386b666-b247-4f7e-be58-b2032d04c662 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:19.678657 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d386b666-b247-4f7e-be58-b2032d04c662 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:19.678932 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d386b666-b247-4f7e-be58-b2032d04c662 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:19.679590 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d386b666-b247-4f7e-be58-b2032d04c662 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:19.681211 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d386b666-b247-4f7e-be58-b2032d04c662 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:19.692230 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1276/2559] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:14:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:14:19.696082 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370ec60e-58ef-4d88-be78-92f2d4bc52bb admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:19.705839 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370ec60e-58ef-4d88-be78-92f2d4bc52bb admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:19.710392 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfb5a0c-719f-42d6-8459-e8f26b61ab72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:19.711029 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb5a0c-719f-42d6-8459-e8f26b61ab72 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:19.711307 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb5a0c-719f-42d6-8459-e8f26b61ab72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:19.711580 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfb5a0c-719f-42d6-8459-e8f26b61ab72 admin admin] PATH_INFO: `/v3/projects/209faf09c1054d148cff932ae5bd533a/users/918b403097b442a7978500acb03b9876/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:19.725748 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfb5a0c-719f-42d6-8459-e8f26b61ab72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=209faf09c1054d148cff932ae5bd533a, user_id=918b403097b442a7978500acb03b9876, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:19.728426 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfb5a0c-719f-42d6-8459-e8f26b61ab72 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:19.741745 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1284/2560] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 04:14:19 2022] PUT /identity/v3/projects/209faf09c1054d148cff932ae5bd533a/users/918b403097b442a7978500acb03b9876/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 04:14:19.753816 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b743b-e6b0-410e-a182-4f62dbef0ee6 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:19.754305 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b743b-e6b0-410e-a182-4f62dbef0ee6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:19.754397 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6b743b-e6b0-410e-a182-4f62dbef0ee6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:19.782213 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-ec6b743b-e6b0-410e-a182-4f62dbef0ee6 None None] MFA Rules not processed for user `918b403097b442a7978500acb03b9876`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:14:19.783473 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ec6b743b-e6b0-410e-a182-4f62dbef0ee6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:14:19.847120 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1277/2561] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:14:19 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:14:19.851515 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfb5a0c-719f-42d6-8459-e8f26b61ab72 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:19.913527 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfb5a0c-719f-42d6-8459-e8f26b61ab72 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:19.918108 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f17ea90-9a63-40eb-a9e7-25a1b4fde023 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:19.918706 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f17ea90-9a63-40eb-a9e7-25a1b4fde023 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:19.918987 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f17ea90-9a63-40eb-a9e7-25a1b4fde023 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:19.919262 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f17ea90-9a63-40eb-a9e7-25a1b4fde023 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:19.919929 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f17ea90-9a63-40eb-a9e7-25a1b4fde023 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:19.921857 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f17ea90-9a63-40eb-a9e7-25a1b4fde023 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:19.949960 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1285/2562] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 04:14:19 2022] POST /identity/v3/projects => generated 393 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:14:19.954005 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6b743b-e6b0-410e-a182-4f62dbef0ee6 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:19.964782 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6b743b-e6b0-410e-a182-4f62dbef0ee6 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:20.020586 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bc98a8-c8bf-4a4e-85d7-6496b362a65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:20.021183 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc98a8-c8bf-4a4e-85d7-6496b362a65f admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:20.021467 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc98a8-c8bf-4a4e-85d7-6496b362a65f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:20.021739 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc98a8-c8bf-4a4e-85d7-6496b362a65f admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:20.022370 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bc98a8-c8bf-4a4e-85d7-6496b362a65f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:20.024404 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bc98a8-c8bf-4a4e-85d7-6496b362a65f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:20.045003 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1278/2563] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 04:14:19 2022] POST /identity/v3/users => generated 303 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 04:14:20.049463 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f17ea90-9a63-40eb-a9e7-25a1b4fde023 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:20.059501 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f17ea90-9a63-40eb-a9e7-25a1b4fde023 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:20.064083 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e1ad3b-a232-4749-a5c1-e34cbdfa5061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:20.064683 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e1ad3b-a232-4749-a5c1-e34cbdfa5061 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:20.064962 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e1ad3b-a232-4749-a5c1-e34cbdfa5061 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:20.065235 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e1ad3b-a232-4749-a5c1-e34cbdfa5061 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:20.065874 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e1ad3b-a232-4749-a5c1-e34cbdfa5061 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:20.067494 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e1ad3b-a232-4749-a5c1-e34cbdfa5061 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:20.077897 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1286/2564] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:14:20.081696 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bc98a8-c8bf-4a4e-85d7-6496b362a65f admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:20.092083 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bc98a8-c8bf-4a4e-85d7-6496b362a65f admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:20.097185 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38ef3ed-fbef-4979-bf40-9a6dc52e3fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:20.097825 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ef3ed-fbef-4979-bf40-9a6dc52e3fed admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:20.098103 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ef3ed-fbef-4979-bf40-9a6dc52e3fed admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:20.098373 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38ef3ed-fbef-4979-bf40-9a6dc52e3fed admin admin] PATH_INFO: `/v3/projects/eab1558c2718495d88b021d97ce1dc1a/users/8f4970708e6c46e4869b681ed242260a/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:20.114517 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38ef3ed-fbef-4979-bf40-9a6dc52e3fed admin admin] RBAC: Authorizing `identity:create_grant(project_id=eab1558c2718495d88b021d97ce1dc1a, user_id=8f4970708e6c46e4869b681ed242260a, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:20.117184 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38ef3ed-fbef-4979-bf40-9a6dc52e3fed admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:20.131117 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1279/2565] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 04:14:20 2022] PUT /identity/v3/projects/eab1558c2718495d88b021d97ce1dc1a/users/8f4970708e6c46e4869b681ed242260a/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 04:14:20.135058 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e1ad3b-a232-4749-a5c1-e34cbdfa5061 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:20.145121 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e1ad3b-a232-4749-a5c1-e34cbdfa5061 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:20.165331 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:14:20.203500 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d20297-c7ad-4978-a783-aeac6e9da906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:20.204134 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d20297-c7ad-4978-a783-aeac6e9da906 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:20.204415 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d20297-c7ad-4978-a783-aeac6e9da906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:20.204689 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d20297-c7ad-4978-a783-aeac6e9da906 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:20.205346 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d20297-c7ad-4978-a783-aeac6e9da906 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:20.206931 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89d20297-c7ad-4978-a783-aeac6e9da906 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:20.216932 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1287/2566] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:14:20 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:14:20.221561 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38ef3ed-fbef-4979-bf40-9a6dc52e3fed admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:20.226910 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:14:20.232988 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38ef3ed-fbef-4979-bf40-9a6dc52e3fed admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:20.238290 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb30b8ec-69e1-4f74-bb53-cb76fef1153c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:20.238937 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb30b8ec-69e1-4f74-bb53-cb76fef1153c admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:20.239213 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb30b8ec-69e1-4f74-bb53-cb76fef1153c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:20.239484 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb30b8ec-69e1-4f74-bb53-cb76fef1153c admin admin] PATH_INFO: `/v3/projects/eab1558c2718495d88b021d97ce1dc1a/users/8f4970708e6c46e4869b681ed242260a/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:20.241369 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb30b8ec-69e1-4f74-bb53-cb76fef1153c admin admin] RBAC: Authorizing `identity:create_grant(project_id=eab1558c2718495d88b021d97ce1dc1a, user_id=8f4970708e6c46e4869b681ed242260a, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:20.243933 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb30b8ec-69e1-4f74-bb53-cb76fef1153c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:20.254814 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1280/2567] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 04:14:20 2022] PUT /identity/v3/projects/eab1558c2718495d88b021d97ce1dc1a/users/8f4970708e6c46e4869b681ed242260a/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 04:14:20.264914 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0385f3d-47ec-48f6-9346-8ca2a8612c73 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:20.265231 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0385f3d-47ec-48f6-9346-8ca2a8612c73 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:20.265509 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0385f3d-47ec-48f6-9346-8ca2a8612c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:20.293046 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-b0385f3d-47ec-48f6-9346-8ca2a8612c73 None None] MFA Rules not processed for user `8f4970708e6c46e4869b681ed242260a`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:14:20.294277 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-b0385f3d-47ec-48f6-9346-8ca2a8612c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:14:20.362103 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1288/2568] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:14:20 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 04:14:20.372832 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb30b8ec-69e1-4f74-bb53-cb76fef1153c admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:20.374545 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-bb30b8ec-69e1-4f74-bb53-cb76fef1153c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:14:20.434355 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb30b8ec-69e1-4f74-bb53-cb76fef1153c admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:20.437872 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cd21f4-231e-41a1-9616-f17f38d6941d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:20.438480 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd21f4-231e-41a1-9616-f17f38d6941d service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:20.438754 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd21f4-231e-41a1-9616-f17f38d6941d service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:20.439572 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cd21f4-231e-41a1-9616-f17f38d6941d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:20.448449 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cd21f4-231e-41a1-9616-f17f38d6941d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:20.449865 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cd21f4-231e-41a1-9616-f17f38d6941d service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:20.459063 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1281/2569] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:14:20 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:14:20.507384 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0385f3d-47ec-48f6-9346-8ca2a8612c73 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:20.508986 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-b0385f3d-47ec-48f6-9346-8ca2a8612c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:14:20.521219 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0385f3d-47ec-48f6-9346-8ca2a8612c73 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:20.524978 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a25e9be-db22-4b93-954b-5b23d1e54ee0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:20.525587 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a25e9be-db22-4b93-954b-5b23d1e54ee0 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:20.525861 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a25e9be-db22-4b93-954b-5b23d1e54ee0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:20.526189 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a25e9be-db22-4b93-954b-5b23d1e54ee0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:20.582615 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a25e9be-db22-4b93-954b-5b23d1e54ee0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:20.584101 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a25e9be-db22-4b93-954b-5b23d1e54ee0 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:20.593317 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1289/2570] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:14:20 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:14:23.851430 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cd21f4-231e-41a1-9616-f17f38d6941d service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:23.912794 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cd21f4-231e-41a1-9616-f17f38d6941d service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:23.917366 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28df53e8-fbe2-406e-8bb8-e423e33bcc9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:23.917985 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df53e8-fbe2-406e-8bb8-e423e33bcc9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:23.918265 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df53e8-fbe2-406e-8bb8-e423e33bcc9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:23.918537 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28df53e8-fbe2-406e-8bb8-e423e33bcc9d admin admin] PATH_INFO: `/v3/users/918b403097b442a7978500acb03b9876` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:23.919647 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28df53e8-fbe2-406e-8bb8-e423e33bcc9d admin admin] RBAC: Authorizing `identity:delete_user(user_id=918b403097b442a7978500acb03b9876)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:23.921584 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28df53e8-fbe2-406e-8bb8-e423e33bcc9d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:23.966522 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-28df53e8-fbe2-406e-8bb8-e423e33bcc9d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '918b403097b442a7978500acb03b9876'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:23.973458 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-28df53e8-fbe2-406e-8bb8-e423e33bcc9d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '918b403097b442a7978500acb03b9876'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:23.974114 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-28df53e8-fbe2-406e-8bb8-e423e33bcc9d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '918b403097b442a7978500acb03b9876'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:23.986045 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-28df53e8-fbe2-406e-8bb8-e423e33bcc9d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '918b403097b442a7978500acb03b9876'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:23.993096 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-28df53e8-fbe2-406e-8bb8-e423e33bcc9d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '918b403097b442a7978500acb03b9876'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:23.999244 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1282/2571] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 04:14:23 2022] DELETE /identity/v3/users/918b403097b442a7978500acb03b9876 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:14:24.004251 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da900a83-2d96-4652-ab0f-43380d757234 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:24.004724 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da900a83-2d96-4652-ab0f-43380d757234 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:24.004814 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da900a83-2d96-4652-ab0f-43380d757234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:24.024099 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-da900a83-2d96-4652-ab0f-43380d757234 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:14:24.025273 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-da900a83-2d96-4652-ab0f-43380d757234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:14:24.065959 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:14:24.085482 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1290/2572] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:14:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:14:24.095601 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28df53e8-fbe2-406e-8bb8-e423e33bcc9d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:24.097010 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-28df53e8-fbe2-406e-8bb8-e423e33bcc9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:14:24.110347 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:14:24.156116 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28df53e8-fbe2-406e-8bb8-e423e33bcc9d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:24.159705 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a149c350-9176-4360-8db8-abc3e57dfbdd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:24.160336 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149c350-9176-4360-8db8-abc3e57dfbdd service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:24.160613 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149c350-9176-4360-8db8-abc3e57dfbdd service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:24.160882 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a149c350-9176-4360-8db8-abc3e57dfbdd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:24.169927 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a149c350-9176-4360-8db8-abc3e57dfbdd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:24.171403 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a149c350-9176-4360-8db8-abc3e57dfbdd service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:24.181430 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1283/2573] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:14:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:14:24.455321 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da900a83-2d96-4652-ab0f-43380d757234 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:24.456914 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-da900a83-2d96-4652-ab0f-43380d757234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:14:24.474319 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da900a83-2d96-4652-ab0f-43380d757234 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:24.481650 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f100ac9c-0be0-43d5-8371-e90ad3884ae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:24.482289 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f100ac9c-0be0-43d5-8371-e90ad3884ae1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:24.482671 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f100ac9c-0be0-43d5-8371-e90ad3884ae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:24.482877 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f100ac9c-0be0-43d5-8371-e90ad3884ae1 admin admin] PATH_INFO: `/v3/projects/209faf09c1054d148cff932ae5bd533a` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:24.484074 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f100ac9c-0be0-43d5-8371-e90ad3884ae1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=209faf09c1054d148cff932ae5bd533a)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:24.487261 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f100ac9c-0be0-43d5-8371-e90ad3884ae1 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:24.534657 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f100ac9c-0be0-43d5-8371-e90ad3884ae1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '209faf09c1054d148cff932ae5bd533a', 'request_id': 'req-f100ac9c-0be0-43d5-8371-e90ad3884ae1'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:24.540584 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f100ac9c-0be0-43d5-8371-e90ad3884ae1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '209faf09c1054d148cff932ae5bd533a', 'request_id': 'req-f100ac9c-0be0-43d5-8371-e90ad3884ae1'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:24.546867 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f100ac9c-0be0-43d5-8371-e90ad3884ae1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '209faf09c1054d148cff932ae5bd533a', 'request_id': 'req-f100ac9c-0be0-43d5-8371-e90ad3884ae1'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:24.559673 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f100ac9c-0be0-43d5-8371-e90ad3884ae1 admin admin] The token cache is being invalidate because project 209faf09c1054d148cff932ae5bd533a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 04:14:24.559968 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f100ac9c-0be0-43d5-8371-e90ad3884ae1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:24.560639 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-f100ac9c-0be0-43d5-8371-e90ad3884ae1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:24.562179 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1291/2574] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 04:14:24 2022] DELETE /identity/v3/projects/209faf09c1054d148cff932ae5bd533a => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:14:24.566159 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a149c350-9176-4360-8db8-abc3e57dfbdd service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:24.575792 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a149c350-9176-4360-8db8-abc3e57dfbdd service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:24.631818 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77daff94-ee23-4869-9ec9-b25cd70c0df7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:24.632473 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77daff94-ee23-4869-9ec9-b25cd70c0df7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:24.632748 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77daff94-ee23-4869-9ec9-b25cd70c0df7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:24.633018 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77daff94-ee23-4869-9ec9-b25cd70c0df7 admin admin] PATH_INFO: `/v3/users/8f4970708e6c46e4869b681ed242260a` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:24.634114 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77daff94-ee23-4869-9ec9-b25cd70c0df7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f4970708e6c46e4869b681ed242260a)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:24.636090 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77daff94-ee23-4869-9ec9-b25cd70c0df7 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:24.680305 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-77daff94-ee23-4869-9ec9-b25cd70c0df7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f4970708e6c46e4869b681ed242260a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:24.687107 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-77daff94-ee23-4869-9ec9-b25cd70c0df7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f4970708e6c46e4869b681ed242260a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:24.687756 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-77daff94-ee23-4869-9ec9-b25cd70c0df7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f4970708e6c46e4869b681ed242260a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:24.699386 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-77daff94-ee23-4869-9ec9-b25cd70c0df7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f4970708e6c46e4869b681ed242260a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:24.706975 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-77daff94-ee23-4869-9ec9-b25cd70c0df7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f4970708e6c46e4869b681ed242260a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:24.712998 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1284/2575] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 04:14:24 2022] DELETE /identity/v3/users/8f4970708e6c46e4869b681ed242260a => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:14:24.987031 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f100ac9c-0be0-43d5-8371-e90ad3884ae1 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:24.988509 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-f100ac9c-0be0-43d5-8371-e90ad3884ae1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:14:25.002220 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f100ac9c-0be0-43d5-8371-e90ad3884ae1 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:25.061626 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719acdc8-65d3-4f9f-8ccf-67e57f2e7ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:25.062264 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719acdc8-65d3-4f9f-8ccf-67e57f2e7ff8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:25.062438 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719acdc8-65d3-4f9f-8ccf-67e57f2e7ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:25.062712 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719acdc8-65d3-4f9f-8ccf-67e57f2e7ff8 admin admin] PATH_INFO: `/v3/projects/eab1558c2718495d88b021d97ce1dc1a` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:25.063842 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719acdc8-65d3-4f9f-8ccf-67e57f2e7ff8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eab1558c2718495d88b021d97ce1dc1a)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:25.065805 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719acdc8-65d3-4f9f-8ccf-67e57f2e7ff8 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:25.111455 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-719acdc8-65d3-4f9f-8ccf-67e57f2e7ff8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eab1558c2718495d88b021d97ce1dc1a', 'request_id': 'req-719acdc8-65d3-4f9f-8ccf-67e57f2e7ff8'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:25.117084 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-719acdc8-65d3-4f9f-8ccf-67e57f2e7ff8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eab1558c2718495d88b021d97ce1dc1a', 'request_id': 'req-719acdc8-65d3-4f9f-8ccf-67e57f2e7ff8'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:25.123548 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-719acdc8-65d3-4f9f-8ccf-67e57f2e7ff8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eab1558c2718495d88b021d97ce1dc1a', 'request_id': 'req-719acdc8-65d3-4f9f-8ccf-67e57f2e7ff8'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:25.132137 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:14:25.137314 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-719acdc8-65d3-4f9f-8ccf-67e57f2e7ff8 admin admin] The token cache is being invalidate because project eab1558c2718495d88b021d97ce1dc1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 04:14:25.137800 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-719acdc8-65d3-4f9f-8ccf-67e57f2e7ff8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:25.138904 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-719acdc8-65d3-4f9f-8ccf-67e57f2e7ff8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:25.140439 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1292/2576] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 04:14:24 2022] DELETE /identity/v3/projects/eab1558c2718495d88b021d97ce1dc1a => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:14:25.145789 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5df9ad-7b79-4ee9-a2a2-b5b286c0df63 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:25.146250 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5df9ad-7b79-4ee9-a2a2-b5b286c0df63 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:25.146374 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5df9ad-7b79-4ee9-a2a2-b5b286c0df63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:25.165411 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-aa5df9ad-7b79-4ee9-a2a2-b5b286c0df63 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:14:25.166593 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-aa5df9ad-7b79-4ee9-a2a2-b5b286c0df63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:14:25.195757 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:14:25.227446 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1285/2577] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:14:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:14:25.236611 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35258bba-1c8a-4909-8ccc-1f6135c3b45b None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:25.236902 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35258bba-1c8a-4909-8ccc-1f6135c3b45b None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:25.237208 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35258bba-1c8a-4909-8ccc-1f6135c3b45b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:25.256798 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-35258bba-1c8a-4909-8ccc-1f6135c3b45b None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:14:25.257995 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-35258bba-1c8a-4909-8ccc-1f6135c3b45b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:14:25.265781 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1293/2578] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:14:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:14:25.269873 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5df9ad-7b79-4ee9-a2a2-b5b286c0df63 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:25.279541 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5df9ad-7b79-4ee9-a2a2-b5b286c0df63 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:25.284611 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d52df7-cfde-4399-adc3-641c2eb1bb51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:25.285207 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d52df7-cfde-4399-adc3-641c2eb1bb51 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:25.285489 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d52df7-cfde-4399-adc3-641c2eb1bb51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:25.285764 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d52df7-cfde-4399-adc3-641c2eb1bb51 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:25.286405 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d52df7-cfde-4399-adc3-641c2eb1bb51 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 04:14:25.286707 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d52df7-cfde-4399-adc3-641c2eb1bb51 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:25.288341 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d52df7-cfde-4399-adc3-641c2eb1bb51 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:25.300432 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1286/2579] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 04:14:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 04:14:25.304476 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35258bba-1c8a-4909-8ccc-1f6135c3b45b None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:25.313698 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35258bba-1c8a-4909-8ccc-1f6135c3b45b None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:25.318390 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c818f563-f9c9-4faf-8ca2-c52276587caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:25.318990 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c818f563-f9c9-4faf-8ca2-c52276587caf admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:25.319274 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c818f563-f9c9-4faf-8ca2-c52276587caf admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:25.319545 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c818f563-f9c9-4faf-8ca2-c52276587caf admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:25.320209 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c818f563-f9c9-4faf-8ca2-c52276587caf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:25.322100 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c818f563-f9c9-4faf-8ca2-c52276587caf admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:25.349161 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1294/2580] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 04:14:25 2022] POST /identity/v3/projects => generated 379 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:14:25.353324 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d52df7-cfde-4399-adc3-641c2eb1bb51 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:25.363772 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d52df7-cfde-4399-adc3-641c2eb1bb51 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:25.418826 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0567c223-ea74-4d5f-a583-61748179ed76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:25.419458 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0567c223-ea74-4d5f-a583-61748179ed76 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:25.419738 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0567c223-ea74-4d5f-a583-61748179ed76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:25.420050 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0567c223-ea74-4d5f-a583-61748179ed76 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:25.420687 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0567c223-ea74-4d5f-a583-61748179ed76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:25.422591 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0567c223-ea74-4d5f-a583-61748179ed76 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:25.446777 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1287/2581] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 04:14:25 2022] POST /identity/v3/users => generated 290 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:14:25.448003 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c818f563-f9c9-4faf-8ca2-c52276587caf admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:25.458302 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c818f563-f9c9-4faf-8ca2-c52276587caf admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:25.462864 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b0cd22-65a7-4e02-b9a3-603c8ba7991b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:25.463460 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0cd22-65a7-4e02-b9a3-603c8ba7991b admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:25.463740 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0cd22-65a7-4e02-b9a3-603c8ba7991b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:25.464047 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0cd22-65a7-4e02-b9a3-603c8ba7991b admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:25.464694 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b0cd22-65a7-4e02-b9a3-603c8ba7991b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:25.466273 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b0cd22-65a7-4e02-b9a3-603c8ba7991b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:25.476589 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1295/2582] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:14:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:14:25.480773 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0567c223-ea74-4d5f-a583-61748179ed76 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:25.491612 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0567c223-ea74-4d5f-a583-61748179ed76 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:25.496321 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c940f3f6-b42c-49fb-a1c6-0270cd970cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:25.496917 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c940f3f6-b42c-49fb-a1c6-0270cd970cf2 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:25.497226 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c940f3f6-b42c-49fb-a1c6-0270cd970cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:25.497755 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c940f3f6-b42c-49fb-a1c6-0270cd970cf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:25.498395 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c940f3f6-b42c-49fb-a1c6-0270cd970cf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:25.500105 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c940f3f6-b42c-49fb-a1c6-0270cd970cf2 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:25.510587 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1288/2583] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:14:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Nov 02 04:14:25.514634 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b0cd22-65a7-4e02-b9a3-603c8ba7991b admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:25.524584 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b0cd22-65a7-4e02-b9a3-603c8ba7991b admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:25.529472 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01a2340-7813-42ed-919b-7bb4bd790252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:25.529995 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01a2340-7813-42ed-919b-7bb4bd790252 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:25.530327 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01a2340-7813-42ed-919b-7bb4bd790252 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:25.530615 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01a2340-7813-42ed-919b-7bb4bd790252 admin admin] PATH_INFO: `/v3/projects/31abd05dd6c94d01b83c9ada6125d979/users/b72371efd70b46968ad3b998e225dda7/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:25.546186 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01a2340-7813-42ed-919b-7bb4bd790252 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31abd05dd6c94d01b83c9ada6125d979, user_id=b72371efd70b46968ad3b998e225dda7, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:25.549491 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01a2340-7813-42ed-919b-7bb4bd790252 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:25.561883 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1296/2584] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 04:14:25 2022] PUT /identity/v3/projects/31abd05dd6c94d01b83c9ada6125d979/users/b72371efd70b46968ad3b998e225dda7/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:14:25.572266 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de7308-d4c6-4c7e-abe6-77f93cbff4ba None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:25.572609 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de7308-d4c6-4c7e-abe6-77f93cbff4ba None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:25.572959 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62de7308-d4c6-4c7e-abe6-77f93cbff4ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:25.602707 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-62de7308-d4c6-4c7e-abe6-77f93cbff4ba None None] MFA Rules not processed for user `b72371efd70b46968ad3b998e225dda7`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:14:25.603937 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-62de7308-d4c6-4c7e-abe6-77f93cbff4ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:14:25.666764 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1289/2585] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:14:25 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:14:25.671134 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01a2340-7813-42ed-919b-7bb4bd790252 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:25.730448 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01a2340-7813-42ed-919b-7bb4bd790252 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:25.734997 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b62e66f-5b53-4b28-8efd-8ff84ec16a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:25.735610 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b62e66f-5b53-4b28-8efd-8ff84ec16a59 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:25.735893 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b62e66f-5b53-4b28-8efd-8ff84ec16a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:25.736191 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b62e66f-5b53-4b28-8efd-8ff84ec16a59 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:25.736809 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b62e66f-5b53-4b28-8efd-8ff84ec16a59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:25.738677 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b62e66f-5b53-4b28-8efd-8ff84ec16a59 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:25.763756 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1297/2586] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 04:14:25 2022] POST /identity/v3/projects => generated 381 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:14:25.768285 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62de7308-d4c6-4c7e-abe6-77f93cbff4ba None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:25.778891 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62de7308-d4c6-4c7e-abe6-77f93cbff4ba None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:25.804642 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:14:25.835851 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1992832c-3d46-40ce-845e-a0b0c3927908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:25.836424 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992832c-3d46-40ce-845e-a0b0c3927908 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:25.836704 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992832c-3d46-40ce-845e-a0b0c3927908 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:25.836977 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1992832c-3d46-40ce-845e-a0b0c3927908 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:25.837616 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1992832c-3d46-40ce-845e-a0b0c3927908 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:25.839517 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1992832c-3d46-40ce-845e-a0b0c3927908 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:25.859536 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1290/2587] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 04:14:25 2022] POST /identity/v3/users => generated 297 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:14:25.863531 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b62e66f-5b53-4b28-8efd-8ff84ec16a59 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:25.874041 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b62e66f-5b53-4b28-8efd-8ff84ec16a59 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:25.878624 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f31402-129a-4b41-9fa6-a97a5dc184c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:25.879244 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f31402-129a-4b41-9fa6-a97a5dc184c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:25.879525 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f31402-129a-4b41-9fa6-a97a5dc184c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:25.879797 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f31402-129a-4b41-9fa6-a97a5dc184c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:25.880468 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f31402-129a-4b41-9fa6-a97a5dc184c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:25.882058 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f31402-129a-4b41-9fa6-a97a5dc184c7 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:25.892919 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1298/2588] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:14:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:14:25.896844 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1992832c-3d46-40ce-845e-a0b0c3927908 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:25.906671 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1992832c-3d46-40ce-845e-a0b0c3927908 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:25.911968 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa413b8e-19fb-4705-814c-42a236b066a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:25.912626 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa413b8e-19fb-4705-814c-42a236b066a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:25.912906 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa413b8e-19fb-4705-814c-42a236b066a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:25.913181 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa413b8e-19fb-4705-814c-42a236b066a6 admin admin] PATH_INFO: `/v3/projects/df30a5c163264106b3deb0f5ce3de234/users/39c610d094c74290af710f10aa4f3aa3/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:25.927590 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa413b8e-19fb-4705-814c-42a236b066a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df30a5c163264106b3deb0f5ce3de234, user_id=39c610d094c74290af710f10aa4f3aa3, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:25.930561 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa413b8e-19fb-4705-814c-42a236b066a6 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:25.943981 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1291/2589] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 04:14:25 2022] PUT /identity/v3/projects/df30a5c163264106b3deb0f5ce3de234/users/39c610d094c74290af710f10aa4f3aa3/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:14:25.947923 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f31402-129a-4b41-9fa6-a97a5dc184c7 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:25.958317 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f31402-129a-4b41-9fa6-a97a5dc184c7 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:26.014492 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af5b132-72c8-48a3-bcf8-b2f08f6da866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:26.015084 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5b132-72c8-48a3-bcf8-b2f08f6da866 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:26.015364 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5b132-72c8-48a3-bcf8-b2f08f6da866 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:26.015637 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af5b132-72c8-48a3-bcf8-b2f08f6da866 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:26.016308 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af5b132-72c8-48a3-bcf8-b2f08f6da866 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:26.018574 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af5b132-72c8-48a3-bcf8-b2f08f6da866 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:26.028157 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1299/2590] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:14:25 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:14:26.031962 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa413b8e-19fb-4705-814c-42a236b066a6 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:26.042237 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa413b8e-19fb-4705-814c-42a236b066a6 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:26.047084 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fcbc6f-18ff-48d7-9674-186a9ae0f690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:26.047767 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fcbc6f-18ff-48d7-9674-186a9ae0f690 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:26.048084 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fcbc6f-18ff-48d7-9674-186a9ae0f690 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:26.048362 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fcbc6f-18ff-48d7-9674-186a9ae0f690 admin admin] PATH_INFO: `/v3/projects/df30a5c163264106b3deb0f5ce3de234/users/39c610d094c74290af710f10aa4f3aa3/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:26.050280 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fcbc6f-18ff-48d7-9674-186a9ae0f690 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df30a5c163264106b3deb0f5ce3de234, user_id=39c610d094c74290af710f10aa4f3aa3, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:26.052959 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fcbc6f-18ff-48d7-9674-186a9ae0f690 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:26.063009 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1292/2591] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 04:14:26 2022] PUT /identity/v3/projects/df30a5c163264106b3deb0f5ce3de234/users/39c610d094c74290af710f10aa4f3aa3/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:14:26.073666 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f9d22-e0b0-4e4c-9449-0bfaa9f135c2 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:26.074132 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f9d22-e0b0-4e4c-9449-0bfaa9f135c2 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:26.074245 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7f9d22-e0b0-4e4c-9449-0bfaa9f135c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:26.082869 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:14:26.103791 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-1e7f9d22-e0b0-4e4c-9449-0bfaa9f135c2 None None] MFA Rules not processed for user `39c610d094c74290af710f10aa4f3aa3`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:14:26.104977 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-1e7f9d22-e0b0-4e4c-9449-0bfaa9f135c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:14:26.173001 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1300/2592] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:14:26 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:14:26.183425 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fcbc6f-18ff-48d7-9674-186a9ae0f690 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:26.185152 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-25fcbc6f-18ff-48d7-9674-186a9ae0f690 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:14:26.241177 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fcbc6f-18ff-48d7-9674-186a9ae0f690 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:26.244752 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47de031-a24f-4be1-94b9-60b5dd3eac93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:26.245341 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47de031-a24f-4be1-94b9-60b5dd3eac93 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:26.245618 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47de031-a24f-4be1-94b9-60b5dd3eac93 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:26.245886 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47de031-a24f-4be1-94b9-60b5dd3eac93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:26.255485 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47de031-a24f-4be1-94b9-60b5dd3eac93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:26.256918 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47de031-a24f-4be1-94b9-60b5dd3eac93 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:26.265992 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1293/2593] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:14:26 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:14:26.367229 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7f9d22-e0b0-4e4c-9449-0bfaa9f135c2 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:26.377655 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7f9d22-e0b0-4e4c-9449-0bfaa9f135c2 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:26.381240 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf136086-be8b-4165-a158-cfa1c676fd21 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:26.381844 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf136086-be8b-4165-a158-cfa1c676fd21 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:26.382121 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf136086-be8b-4165-a158-cfa1c676fd21 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:26.382394 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf136086-be8b-4165-a158-cfa1c676fd21 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:26.441061 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf136086-be8b-4165-a158-cfa1c676fd21 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:26.442498 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf136086-be8b-4165-a158-cfa1c676fd21 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:26.451437 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1301/2594] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:14:26 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Nov 02 04:14:29.027671 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47de031-a24f-4be1-94b9-60b5dd3eac93 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:29.088821 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47de031-a24f-4be1-94b9-60b5dd3eac93 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:29.093384 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d14bb84-7a7f-4e9a-9de8-25ae3148fa5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:29.094011 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d14bb84-7a7f-4e9a-9de8-25ae3148fa5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:29.094293 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d14bb84-7a7f-4e9a-9de8-25ae3148fa5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:29.094567 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d14bb84-7a7f-4e9a-9de8-25ae3148fa5f admin admin] PATH_INFO: `/v3/users/b72371efd70b46968ad3b998e225dda7` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:29.095697 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d14bb84-7a7f-4e9a-9de8-25ae3148fa5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b72371efd70b46968ad3b998e225dda7)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:29.097648 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d14bb84-7a7f-4e9a-9de8-25ae3148fa5f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:29.124527 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:14:29.145784 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8d14bb84-7a7f-4e9a-9de8-25ae3148fa5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b72371efd70b46968ad3b998e225dda7'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:29.152899 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8d14bb84-7a7f-4e9a-9de8-25ae3148fa5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b72371efd70b46968ad3b998e225dda7'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:29.153560 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8d14bb84-7a7f-4e9a-9de8-25ae3148fa5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b72371efd70b46968ad3b998e225dda7'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:29.165621 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8d14bb84-7a7f-4e9a-9de8-25ae3148fa5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b72371efd70b46968ad3b998e225dda7'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:29.172511 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8d14bb84-7a7f-4e9a-9de8-25ae3148fa5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b72371efd70b46968ad3b998e225dda7'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:29.178464 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1294/2595] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 04:14:29 2022] DELETE /identity/v3/users/b72371efd70b46968ad3b998e225dda7 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:14:29.182921 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d774f2-c51a-45cb-8d94-a2bd6d12467c None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:29.183391 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d774f2-c51a-45cb-8d94-a2bd6d12467c None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:29.183485 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d774f2-c51a-45cb-8d94-a2bd6d12467c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:29.202684 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-48d774f2-c51a-45cb-8d94-a2bd6d12467c None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:14:29.203876 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-48d774f2-c51a-45cb-8d94-a2bd6d12467c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:14:29.261746 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1302/2596] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:14:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Nov 02 04:14:29.272064 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d14bb84-7a7f-4e9a-9de8-25ae3148fa5f admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:29.273468 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8d14bb84-7a7f-4e9a-9de8-25ae3148fa5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:14:29.330346 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d14bb84-7a7f-4e9a-9de8-25ae3148fa5f admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:29.334028 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e7cd5e-ecfc-433c-a41b-952275cb1a3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:29.334631 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e7cd5e-ecfc-433c-a41b-952275cb1a3b service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:29.334911 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e7cd5e-ecfc-433c-a41b-952275cb1a3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:29.335186 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e7cd5e-ecfc-433c-a41b-952275cb1a3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:29.344305 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e7cd5e-ecfc-433c-a41b-952275cb1a3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:29.345791 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e7cd5e-ecfc-433c-a41b-952275cb1a3b service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:29.355347 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1295/2597] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:14:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:14:29.621097 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d774f2-c51a-45cb-8d94-a2bd6d12467c None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:29.622895 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-48d774f2-c51a-45cb-8d94-a2bd6d12467c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:14:29.636490 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d774f2-c51a-45cb-8d94-a2bd6d12467c None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:29.641702 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ce33c3-0f65-4104-90cf-1535e25fc096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:29.642362 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce33c3-0f65-4104-90cf-1535e25fc096 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:29.642642 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce33c3-0f65-4104-90cf-1535e25fc096 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:29.642920 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce33c3-0f65-4104-90cf-1535e25fc096 admin admin] PATH_INFO: `/v3/projects/31abd05dd6c94d01b83c9ada6125d979` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:29.644086 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ce33c3-0f65-4104-90cf-1535e25fc096 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31abd05dd6c94d01b83c9ada6125d979)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:29.646028 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ce33c3-0f65-4104-90cf-1535e25fc096 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:29.671009 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:14:29.693877 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-18ce33c3-0f65-4104-90cf-1535e25fc096 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31abd05dd6c94d01b83c9ada6125d979', 'request_id': 'req-18ce33c3-0f65-4104-90cf-1535e25fc096'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:29.699412 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-18ce33c3-0f65-4104-90cf-1535e25fc096 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31abd05dd6c94d01b83c9ada6125d979', 'request_id': 'req-18ce33c3-0f65-4104-90cf-1535e25fc096'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:29.705577 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-18ce33c3-0f65-4104-90cf-1535e25fc096 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31abd05dd6c94d01b83c9ada6125d979', 'request_id': 'req-18ce33c3-0f65-4104-90cf-1535e25fc096'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:29.717323 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-18ce33c3-0f65-4104-90cf-1535e25fc096 admin admin] The token cache is being invalidate because project 31abd05dd6c94d01b83c9ada6125d979 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 04:14:29.717599 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-18ce33c3-0f65-4104-90cf-1535e25fc096 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:29.718218 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-18ce33c3-0f65-4104-90cf-1535e25fc096 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:29.719686 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1303/2598] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 04:14:29 2022] DELETE /identity/v3/projects/31abd05dd6c94d01b83c9ada6125d979 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:14:29.724155 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e7cd5e-ecfc-433c-a41b-952275cb1a3b service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:29.733739 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e7cd5e-ecfc-433c-a41b-952275cb1a3b service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:29.789950 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f734281b-756d-495d-a1c7-471e9a7d42f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:29.790581 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f734281b-756d-495d-a1c7-471e9a7d42f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:29.790863 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f734281b-756d-495d-a1c7-471e9a7d42f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:29.791138 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f734281b-756d-495d-a1c7-471e9a7d42f8 admin admin] PATH_INFO: `/v3/users/39c610d094c74290af710f10aa4f3aa3` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:29.792295 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f734281b-756d-495d-a1c7-471e9a7d42f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=39c610d094c74290af710f10aa4f3aa3)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:29.794216 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f734281b-756d-495d-a1c7-471e9a7d42f8 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:29.840737 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f734281b-756d-495d-a1c7-471e9a7d42f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '39c610d094c74290af710f10aa4f3aa3'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:29.841929 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:14:29.850454 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f734281b-756d-495d-a1c7-471e9a7d42f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '39c610d094c74290af710f10aa4f3aa3'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:29.851316 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f734281b-756d-495d-a1c7-471e9a7d42f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '39c610d094c74290af710f10aa4f3aa3'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:29.863968 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f734281b-756d-495d-a1c7-471e9a7d42f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '39c610d094c74290af710f10aa4f3aa3'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:29.871481 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f734281b-756d-495d-a1c7-471e9a7d42f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '39c610d094c74290af710f10aa4f3aa3'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:29.877751 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1296/2599] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 04:14:29 2022] DELETE /identity/v3/users/39c610d094c74290af710f10aa4f3aa3 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:14:30.145952 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ce33c3-0f65-4104-90cf-1535e25fc096 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:30.147541 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-18ce33c3-0f65-4104-90cf-1535e25fc096 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:14:30.161384 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ce33c3-0f65-4104-90cf-1535e25fc096 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:30.217483 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89844706-7b14-4a76-9ba1-45a4042184bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:30.218147 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89844706-7b14-4a76-9ba1-45a4042184bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:30.218429 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89844706-7b14-4a76-9ba1-45a4042184bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:30.218757 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89844706-7b14-4a76-9ba1-45a4042184bb admin admin] PATH_INFO: `/v3/projects/df30a5c163264106b3deb0f5ce3de234` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:30.219834 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89844706-7b14-4a76-9ba1-45a4042184bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=df30a5c163264106b3deb0f5ce3de234)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:30.222415 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89844706-7b14-4a76-9ba1-45a4042184bb admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:30.267484 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-89844706-7b14-4a76-9ba1-45a4042184bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df30a5c163264106b3deb0f5ce3de234', 'request_id': 'req-89844706-7b14-4a76-9ba1-45a4042184bb'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:30.273073 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-89844706-7b14-4a76-9ba1-45a4042184bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df30a5c163264106b3deb0f5ce3de234', 'request_id': 'req-89844706-7b14-4a76-9ba1-45a4042184bb'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:30.279090 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-89844706-7b14-4a76-9ba1-45a4042184bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df30a5c163264106b3deb0f5ce3de234', 'request_id': 'req-89844706-7b14-4a76-9ba1-45a4042184bb'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:30.291317 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-89844706-7b14-4a76-9ba1-45a4042184bb admin admin] The token cache is being invalidate because project df30a5c163264106b3deb0f5ce3de234 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 04:14:30.291608 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-89844706-7b14-4a76-9ba1-45a4042184bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:30.292255 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-89844706-7b14-4a76-9ba1-45a4042184bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:30.293723 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1304/2600] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 04:14:30 2022] DELETE /identity/v3/projects/df30a5c163264106b3deb0f5ce3de234 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:14:30.298640 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a205403e-7f6e-4469-97f7-167ada6a26cd None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:30.299114 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a205403e-7f6e-4469-97f7-167ada6a26cd None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:30.299330 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a205403e-7f6e-4469-97f7-167ada6a26cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:30.318618 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a205403e-7f6e-4469-97f7-167ada6a26cd None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:14:30.319913 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a205403e-7f6e-4469-97f7-167ada6a26cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:14:30.379842 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1297/2601] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:14:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:14:30.387578 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7749e82e-df68-4542-956f-25784857d5b9 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:30.387870 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7749e82e-df68-4542-956f-25784857d5b9 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:30.388177 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7749e82e-df68-4542-956f-25784857d5b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:30.409203 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-7749e82e-df68-4542-956f-25784857d5b9 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:14:30.410405 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-7749e82e-df68-4542-956f-25784857d5b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:14:30.418318 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1305/2602] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:14:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:14:30.422478 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a205403e-7f6e-4469-97f7-167ada6a26cd None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:30.438041 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a205403e-7f6e-4469-97f7-167ada6a26cd None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:30.442653 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f5edd0-e22a-4b9b-b64b-d0c9a790abdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:30.443257 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5edd0-e22a-4b9b-b64b-d0c9a790abdb admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:30.443535 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5edd0-e22a-4b9b-b64b-d0c9a790abdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:30.443805 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f5edd0-e22a-4b9b-b64b-d0c9a790abdb admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:30.444481 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f5edd0-e22a-4b9b-b64b-d0c9a790abdb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 04:14:30.444783 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f5edd0-e22a-4b9b-b64b-d0c9a790abdb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:30.446393 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f5edd0-e22a-4b9b-b64b-d0c9a790abdb admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:30.458998 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1298/2603] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 04:14:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Nov 02 04:14:30.463306 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7749e82e-df68-4542-956f-25784857d5b9 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:30.473218 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7749e82e-df68-4542-956f-25784857d5b9 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:30.478652 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3810489c-85ea-4c2d-a462-11d827328124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:30.479250 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3810489c-85ea-4c2d-a462-11d827328124 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:30.479528 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3810489c-85ea-4c2d-a462-11d827328124 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:30.479799 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3810489c-85ea-4c2d-a462-11d827328124 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:30.480467 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3810489c-85ea-4c2d-a462-11d827328124 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:30.482349 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3810489c-85ea-4c2d-a462-11d827328124 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:30.510326 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1306/2604] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 04:14:30 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Nov 02 04:14:30.514384 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f5edd0-e22a-4b9b-b64b-d0c9a790abdb admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:30.524692 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f5edd0-e22a-4b9b-b64b-d0c9a790abdb admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:30.581546 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d85c818-a3cc-42d6-b09f-c4f9d9f78d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:30.582153 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d85c818-a3cc-42d6-b09f-c4f9d9f78d9b admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:30.582436 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d85c818-a3cc-42d6-b09f-c4f9d9f78d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:30.582713 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d85c818-a3cc-42d6-b09f-c4f9d9f78d9b admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:30.583344 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d85c818-a3cc-42d6-b09f-c4f9d9f78d9b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:30.585275 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d85c818-a3cc-42d6-b09f-c4f9d9f78d9b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:30.605519 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1299/2605] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 04:14:30 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:14:30.609630 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3810489c-85ea-4c2d-a462-11d827328124 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:30.619743 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3810489c-85ea-4c2d-a462-11d827328124 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:30.624703 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fdace9-a1ca-4f09-a174-910908415e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:30.625304 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fdace9-a1ca-4f09-a174-910908415e00 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:30.625577 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fdace9-a1ca-4f09-a174-910908415e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:30.625848 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fdace9-a1ca-4f09-a174-910908415e00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:30.627019 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fdace9-a1ca-4f09-a174-910908415e00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:30.628647 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14fdace9-a1ca-4f09-a174-910908415e00 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:30.638527 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1307/2606] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:14:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:14:30.642914 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d85c818-a3cc-42d6-b09f-c4f9d9f78d9b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:30.652818 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d85c818-a3cc-42d6-b09f-c4f9d9f78d9b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:30.657515 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5641c8-0d3d-4a03-871a-c8041f398af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:30.658101 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5641c8-0d3d-4a03-871a-c8041f398af4 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:30.658378 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5641c8-0d3d-4a03-871a-c8041f398af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:30.658645 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5641c8-0d3d-4a03-871a-c8041f398af4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:30.659271 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5641c8-0d3d-4a03-871a-c8041f398af4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:30.660896 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5641c8-0d3d-4a03-871a-c8041f398af4 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:30.671347 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1300/2607] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:14:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:14:30.675278 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fdace9-a1ca-4f09-a174-910908415e00 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:30.680379 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:14:30.686353 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fdace9-a1ca-4f09-a174-910908415e00 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:30.692503 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ec7576-57ab-4874-b759-dbdfc825a1b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:30.693141 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec7576-57ab-4874-b759-dbdfc825a1b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:30.693420 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec7576-57ab-4874-b759-dbdfc825a1b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:30.693694 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ec7576-57ab-4874-b759-dbdfc825a1b6 admin admin] PATH_INFO: `/v3/projects/b1e64f14b06a4de1ba0f24c00e3b7e9b/users/8cd250f1a83d4d979f441cae31ae03a3/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:30.708551 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ec7576-57ab-4874-b759-dbdfc825a1b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1e64f14b06a4de1ba0f24c00e3b7e9b, user_id=8cd250f1a83d4d979f441cae31ae03a3, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:30.711183 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ec7576-57ab-4874-b759-dbdfc825a1b6 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:30.723476 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1308/2608] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 04:14:30 2022] PUT /identity/v3/projects/b1e64f14b06a4de1ba0f24c00e3b7e9b/users/8cd250f1a83d4d979f441cae31ae03a3/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:14:30.733503 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44574d9d-6a93-461d-b46a-e28e08e1a03d None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:30.733788 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44574d9d-6a93-461d-b46a-e28e08e1a03d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:30.734056 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44574d9d-6a93-461d-b46a-e28e08e1a03d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:30.762353 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-44574d9d-6a93-461d-b46a-e28e08e1a03d None None] MFA Rules not processed for user `8cd250f1a83d4d979f441cae31ae03a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:14:30.763552 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-44574d9d-6a93-461d-b46a-e28e08e1a03d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:14:30.782172 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:14:30.829034 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1301/2609] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:14:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:14:30.834067 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ec7576-57ab-4874-b759-dbdfc825a1b6 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:30.895759 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ec7576-57ab-4874-b759-dbdfc825a1b6 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:30.900413 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0716ee34-57f0-42d0-881d-861d889427fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:30.901008 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716ee34-57f0-42d0-881d-861d889427fc admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:30.901285 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716ee34-57f0-42d0-881d-861d889427fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:30.901559 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0716ee34-57f0-42d0-881d-861d889427fc admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:30.902177 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0716ee34-57f0-42d0-881d-861d889427fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:30.904085 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0716ee34-57f0-42d0-881d-861d889427fc admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:30.930687 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1309/2610] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 04:14:30 2022] POST /identity/v3/projects => generated 377 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:14:30.934740 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44574d9d-6a93-461d-b46a-e28e08e1a03d None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:30.946225 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44574d9d-6a93-461d-b46a-e28e08e1a03d None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:31.001777 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8548d946-0906-400d-ade7-1771f38a1d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:31.002314 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8548d946-0906-400d-ade7-1771f38a1d17 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:31.002596 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8548d946-0906-400d-ade7-1771f38a1d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:31.002873 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8548d946-0906-400d-ade7-1771f38a1d17 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:31.003506 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8548d946-0906-400d-ade7-1771f38a1d17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:31.005455 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8548d946-0906-400d-ade7-1771f38a1d17 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:31.025419 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1302/2611] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 04:14:30 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:14:31.029232 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0716ee34-57f0-42d0-881d-861d889427fc admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:31.039408 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0716ee34-57f0-42d0-881d-861d889427fc admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:31.043997 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35333739-7b55-4e3c-aaf1-96b749127655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:31.044597 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35333739-7b55-4e3c-aaf1-96b749127655 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:31.044879 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35333739-7b55-4e3c-aaf1-96b749127655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:31.045166 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35333739-7b55-4e3c-aaf1-96b749127655 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:31.045784 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35333739-7b55-4e3c-aaf1-96b749127655 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:31.047369 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35333739-7b55-4e3c-aaf1-96b749127655 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:31.058293 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1310/2612] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:14:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:14:31.062309 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8548d946-0906-400d-ade7-1771f38a1d17 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:31.072752 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8548d946-0906-400d-ade7-1771f38a1d17 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:31.077373 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95492d5-5389-4398-a087-95ef31effd9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:31.077963 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95492d5-5389-4398-a087-95ef31effd9e admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:31.078242 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95492d5-5389-4398-a087-95ef31effd9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:31.078514 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95492d5-5389-4398-a087-95ef31effd9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:31.079148 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95492d5-5389-4398-a087-95ef31effd9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:31.080744 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95492d5-5389-4398-a087-95ef31effd9e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:31.090482 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1303/2613] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:14:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:14:31.094424 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35333739-7b55-4e3c-aaf1-96b749127655 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:31.104211 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35333739-7b55-4e3c-aaf1-96b749127655 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:31.108758 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffe5f9d-03b6-4a45-8414-748b0a00b7f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:31.109394 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe5f9d-03b6-4a45-8414-748b0a00b7f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:31.109671 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe5f9d-03b6-4a45-8414-748b0a00b7f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:31.109942 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffe5f9d-03b6-4a45-8414-748b0a00b7f0 admin admin] PATH_INFO: `/v3/projects/1d66021b7da241fb8ca0a3060f8b7315/users/0b541c7da4bd417b94fc27bbc582b0ce/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:31.124864 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffe5f9d-03b6-4a45-8414-748b0a00b7f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d66021b7da241fb8ca0a3060f8b7315, user_id=0b541c7da4bd417b94fc27bbc582b0ce, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:31.128158 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffe5f9d-03b6-4a45-8414-748b0a00b7f0 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:31.140738 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1311/2614] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 04:14:31 2022] PUT /identity/v3/projects/1d66021b7da241fb8ca0a3060f8b7315/users/0b541c7da4bd417b94fc27bbc582b0ce/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 04:14:31.151523 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06da8b49-57d5-4d54-b121-a3518556ec76 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:31.151838 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06da8b49-57d5-4d54-b121-a3518556ec76 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:31.152148 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06da8b49-57d5-4d54-b121-a3518556ec76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:31.179756 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-06da8b49-57d5-4d54-b121-a3518556ec76 None None] MFA Rules not processed for user `0b541c7da4bd417b94fc27bbc582b0ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:14:31.180995 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-06da8b49-57d5-4d54-b121-a3518556ec76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:14:31.244321 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1304/2615] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:14:31 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:14:31.249045 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffe5f9d-03b6-4a45-8414-748b0a00b7f0 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:31.310002 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffe5f9d-03b6-4a45-8414-748b0a00b7f0 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:31.314852 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37493f2-9ee6-4e89-bd5a-316ba4940b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:31.315452 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37493f2-9ee6-4e89-bd5a-316ba4940b54 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:31.315732 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37493f2-9ee6-4e89-bd5a-316ba4940b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:31.316035 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37493f2-9ee6-4e89-bd5a-316ba4940b54 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:31.316675 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37493f2-9ee6-4e89-bd5a-316ba4940b54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:31.318596 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37493f2-9ee6-4e89-bd5a-316ba4940b54 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:31.344149 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1312/2616] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 04:14:31 2022] POST /identity/v3/projects => generated 379 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:14:31.348700 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06da8b49-57d5-4d54-b121-a3518556ec76 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:31.359640 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06da8b49-57d5-4d54-b121-a3518556ec76 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:31.386057 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:14:31.418046 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20103a34-be5f-4968-baf7-4b91b2b36a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:31.418659 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20103a34-be5f-4968-baf7-4b91b2b36a55 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:31.418938 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20103a34-be5f-4968-baf7-4b91b2b36a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:31.419210 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20103a34-be5f-4968-baf7-4b91b2b36a55 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:31.419837 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20103a34-be5f-4968-baf7-4b91b2b36a55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:31.421768 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20103a34-be5f-4968-baf7-4b91b2b36a55 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:31.443084 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1305/2617] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 04:14:31 2022] POST /identity/v3/users => generated 296 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:14:31.447106 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37493f2-9ee6-4e89-bd5a-316ba4940b54 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:31.458442 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37493f2-9ee6-4e89-bd5a-316ba4940b54 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:31.463814 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d835cab-8101-4811-bcaa-f6a0a66698d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:31.464446 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d835cab-8101-4811-bcaa-f6a0a66698d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:31.464728 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d835cab-8101-4811-bcaa-f6a0a66698d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:31.465022 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d835cab-8101-4811-bcaa-f6a0a66698d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:31.465658 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d835cab-8101-4811-bcaa-f6a0a66698d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:31.467242 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d835cab-8101-4811-bcaa-f6a0a66698d1 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:31.478991 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1313/2618] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:14:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:14:31.483312 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20103a34-be5f-4968-baf7-4b91b2b36a55 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:31.493933 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20103a34-be5f-4968-baf7-4b91b2b36a55 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:31.498899 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c068b0-3f6a-40a2-8e99-db87fa1923cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:31.499545 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c068b0-3f6a-40a2-8e99-db87fa1923cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:31.499822 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c068b0-3f6a-40a2-8e99-db87fa1923cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:31.500139 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c068b0-3f6a-40a2-8e99-db87fa1923cb admin admin] PATH_INFO: `/v3/projects/c7c957a554d940db8f41d811f5d26b7d/users/fb321111efc04caba2c08a6bd31c19eb/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:31.514963 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c068b0-3f6a-40a2-8e99-db87fa1923cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7c957a554d940db8f41d811f5d26b7d, user_id=fb321111efc04caba2c08a6bd31c19eb, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:31.517774 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c068b0-3f6a-40a2-8e99-db87fa1923cb admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:31.533700 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1306/2619] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 04:14:31 2022] PUT /identity/v3/projects/c7c957a554d940db8f41d811f5d26b7d/users/fb321111efc04caba2c08a6bd31c19eb/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:14:31.537725 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d835cab-8101-4811-bcaa-f6a0a66698d1 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:31.543038 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:14:31.550272 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d835cab-8101-4811-bcaa-f6a0a66698d1 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:31.605468 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974cffa9-1fb5-409b-924f-7ad9badd6bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:31.606000 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974cffa9-1fb5-409b-924f-7ad9badd6bc5 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:31.606281 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974cffa9-1fb5-409b-924f-7ad9badd6bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:31.606552 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974cffa9-1fb5-409b-924f-7ad9badd6bc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:31.607175 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974cffa9-1fb5-409b-924f-7ad9badd6bc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:31.608781 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974cffa9-1fb5-409b-924f-7ad9badd6bc5 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:31.618299 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1314/2620] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:14:31 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:14:31.622725 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c068b0-3f6a-40a2-8e99-db87fa1923cb admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:31.633314 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c068b0-3f6a-40a2-8e99-db87fa1923cb admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:31.638125 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7263eac-a63c-4968-a53c-8db86ea47918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:31.638776 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7263eac-a63c-4968-a53c-8db86ea47918 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:31.639052 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7263eac-a63c-4968-a53c-8db86ea47918 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:31.639324 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7263eac-a63c-4968-a53c-8db86ea47918 admin admin] PATH_INFO: `/v3/projects/c7c957a554d940db8f41d811f5d26b7d/users/fb321111efc04caba2c08a6bd31c19eb/roles/addc0db6b4ac4362b07fe21e14f89180` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:31.641241 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7263eac-a63c-4968-a53c-8db86ea47918 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7c957a554d940db8f41d811f5d26b7d, user_id=fb321111efc04caba2c08a6bd31c19eb, role_id=addc0db6b4ac4362b07fe21e14f89180)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:31.644450 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7263eac-a63c-4968-a53c-8db86ea47918 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:31.654611 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1307/2621] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 04:14:31 2022] PUT /identity/v3/projects/c7c957a554d940db8f41d811f5d26b7d/users/fb321111efc04caba2c08a6bd31c19eb/roles/addc0db6b4ac4362b07fe21e14f89180 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:14:31.665126 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89555dd1-8701-4b32-b510-bd93c5258539 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:31.665417 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89555dd1-8701-4b32-b510-bd93c5258539 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:31.665695 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89555dd1-8701-4b32-b510-bd93c5258539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:31.692748 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-89555dd1-8701-4b32-b510-bd93c5258539 None None] MFA Rules not processed for user `fb321111efc04caba2c08a6bd31c19eb`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:14:31.693946 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-89555dd1-8701-4b32-b510-bd93c5258539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:14:31.761098 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1315/2622] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:14:31 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:14:31.772951 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7263eac-a63c-4968-a53c-8db86ea47918 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:31.774692 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e7263eac-a63c-4968-a53c-8db86ea47918 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:14:31.832490 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7263eac-a63c-4968-a53c-8db86ea47918 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:31.836128 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bacef4-ac73-43ee-92e5-ff18e3e31de5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:31.836725 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bacef4-ac73-43ee-92e5-ff18e3e31de5 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:31.837001 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bacef4-ac73-43ee-92e5-ff18e3e31de5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:31.837274 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bacef4-ac73-43ee-92e5-ff18e3e31de5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:31.846832 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bacef4-ac73-43ee-92e5-ff18e3e31de5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:31.848275 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27bacef4-ac73-43ee-92e5-ff18e3e31de5 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:31.857780 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1308/2623] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:14:31 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:14:31.899282 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89555dd1-8701-4b32-b510-bd93c5258539 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:31.900761 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-89555dd1-8701-4b32-b510-bd93c5258539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:14:31.913252 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89555dd1-8701-4b32-b510-bd93c5258539 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:31.916869 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c3344a-c712-43c5-91c1-c8fb7dd68035 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:31.917472 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c3344a-c712-43c5-91c1-c8fb7dd68035 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:31.917753 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c3344a-c712-43c5-91c1-c8fb7dd68035 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:31.918025 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c3344a-c712-43c5-91c1-c8fb7dd68035 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:31.974130 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c3344a-c712-43c5-91c1-c8fb7dd68035 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:31.975559 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c3344a-c712-43c5-91c1-c8fb7dd68035 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:31.984432 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1316/2624] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:14:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:14:34.848334 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bacef4-ac73-43ee-92e5-ff18e3e31de5 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:34.911865 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bacef4-ac73-43ee-92e5-ff18e3e31de5 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:34.916462 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d476c3d-238c-4e11-921c-9b035e03ed30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:34.917088 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d476c3d-238c-4e11-921c-9b035e03ed30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:34.917370 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d476c3d-238c-4e11-921c-9b035e03ed30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:34.917645 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d476c3d-238c-4e11-921c-9b035e03ed30 admin admin] PATH_INFO: `/v3/users/8cd250f1a83d4d979f441cae31ae03a3` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:34.918764 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d476c3d-238c-4e11-921c-9b035e03ed30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cd250f1a83d4d979f441cae31ae03a3)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:34.920717 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d476c3d-238c-4e11-921c-9b035e03ed30 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:34.948128 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:14:34.967836 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7d476c3d-238c-4e11-921c-9b035e03ed30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cd250f1a83d4d979f441cae31ae03a3'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:34.975282 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7d476c3d-238c-4e11-921c-9b035e03ed30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cd250f1a83d4d979f441cae31ae03a3'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:34.975969 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7d476c3d-238c-4e11-921c-9b035e03ed30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cd250f1a83d4d979f441cae31ae03a3'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:34.988263 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7d476c3d-238c-4e11-921c-9b035e03ed30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cd250f1a83d4d979f441cae31ae03a3'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:34.995258 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7d476c3d-238c-4e11-921c-9b035e03ed30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cd250f1a83d4d979f441cae31ae03a3'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:35.001434 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1309/2625] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 04:14:34 2022] DELETE /identity/v3/users/8cd250f1a83d4d979f441cae31ae03a3 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:14:35.006064 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1814bd2d-a17f-461d-9495-970e0cbbe165 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:35.006581 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1814bd2d-a17f-461d-9495-970e0cbbe165 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:35.006684 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1814bd2d-a17f-461d-9495-970e0cbbe165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:35.025324 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-1814bd2d-a17f-461d-9495-970e0cbbe165 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:14:35.026513 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-1814bd2d-a17f-461d-9495-970e0cbbe165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:14:35.043090 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:14:35.084594 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1317/2626] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:14:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:14:35.094275 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d476c3d-238c-4e11-921c-9b035e03ed30 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:35.096336 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7d476c3d-238c-4e11-921c-9b035e03ed30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:14:35.152180 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d476c3d-238c-4e11-921c-9b035e03ed30 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:35.155924 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46d9b61-5d2d-4cae-916f-43644996ff70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:35.156551 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d9b61-5d2d-4cae-916f-43644996ff70 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:35.156835 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d9b61-5d2d-4cae-916f-43644996ff70 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:35.157140 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46d9b61-5d2d-4cae-916f-43644996ff70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:35.166622 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46d9b61-5d2d-4cae-916f-43644996ff70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:35.168078 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46d9b61-5d2d-4cae-916f-43644996ff70 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:35.178062 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1310/2627] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:14:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:14:35.433533 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1814bd2d-a17f-461d-9495-970e0cbbe165 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:35.435156 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-1814bd2d-a17f-461d-9495-970e0cbbe165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:14:35.452323 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1814bd2d-a17f-461d-9495-970e0cbbe165 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:35.457645 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ae8f21-dabb-4560-a8ee-3558ffa4461b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:35.458307 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae8f21-dabb-4560-a8ee-3558ffa4461b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:35.458590 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae8f21-dabb-4560-a8ee-3558ffa4461b admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:35.458865 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ae8f21-dabb-4560-a8ee-3558ffa4461b admin admin] PATH_INFO: `/v3/projects/b1e64f14b06a4de1ba0f24c00e3b7e9b` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:35.460131 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ae8f21-dabb-4560-a8ee-3558ffa4461b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1e64f14b06a4de1ba0f24c00e3b7e9b)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:35.462263 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ae8f21-dabb-4560-a8ee-3558ffa4461b admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:35.509276 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-54ae8f21-dabb-4560-a8ee-3558ffa4461b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1e64f14b06a4de1ba0f24c00e3b7e9b', 'request_id': 'req-54ae8f21-dabb-4560-a8ee-3558ffa4461b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:35.514762 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-54ae8f21-dabb-4560-a8ee-3558ffa4461b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1e64f14b06a4de1ba0f24c00e3b7e9b', 'request_id': 'req-54ae8f21-dabb-4560-a8ee-3558ffa4461b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:35.521586 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-54ae8f21-dabb-4560-a8ee-3558ffa4461b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1e64f14b06a4de1ba0f24c00e3b7e9b', 'request_id': 'req-54ae8f21-dabb-4560-a8ee-3558ffa4461b'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:35.536068 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-54ae8f21-dabb-4560-a8ee-3558ffa4461b admin admin] The token cache is being invalidate because project b1e64f14b06a4de1ba0f24c00e3b7e9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 04:14:35.536362 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-54ae8f21-dabb-4560-a8ee-3558ffa4461b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:35.537001 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-54ae8f21-dabb-4560-a8ee-3558ffa4461b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:35.538514 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1318/2628] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 04:14:35 2022] DELETE /identity/v3/projects/b1e64f14b06a4de1ba0f24c00e3b7e9b => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:14:35.544132 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46d9b61-5d2d-4cae-916f-43644996ff70 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:35.554764 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46d9b61-5d2d-4cae-916f-43644996ff70 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:35.611022 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32550b1f-418f-4a47-b572-31d93f43ae32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:35.611599 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32550b1f-418f-4a47-b572-31d93f43ae32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:35.611880 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32550b1f-418f-4a47-b572-31d93f43ae32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:35.612242 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32550b1f-418f-4a47-b572-31d93f43ae32 admin admin] PATH_INFO: `/v3/users/0b541c7da4bd417b94fc27bbc582b0ce` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:35.613343 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32550b1f-418f-4a47-b572-31d93f43ae32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b541c7da4bd417b94fc27bbc582b0ce)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:35.615585 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32550b1f-418f-4a47-b572-31d93f43ae32 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:35.660753 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-32550b1f-418f-4a47-b572-31d93f43ae32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b541c7da4bd417b94fc27bbc582b0ce'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:35.661913 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:14:35.670009 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-32550b1f-418f-4a47-b572-31d93f43ae32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b541c7da4bd417b94fc27bbc582b0ce'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:35.670745 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-32550b1f-418f-4a47-b572-31d93f43ae32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b541c7da4bd417b94fc27bbc582b0ce'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:35.682525 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-32550b1f-418f-4a47-b572-31d93f43ae32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b541c7da4bd417b94fc27bbc582b0ce'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:35.690761 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-32550b1f-418f-4a47-b572-31d93f43ae32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b541c7da4bd417b94fc27bbc582b0ce'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:35.696690 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1311/2629] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 04:14:35 2022] DELETE /identity/v3/users/0b541c7da4bd417b94fc27bbc582b0ce => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:14:35.965304 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ae8f21-dabb-4560-a8ee-3558ffa4461b admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:35.966809 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-54ae8f21-dabb-4560-a8ee-3558ffa4461b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:14:35.979883 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ae8f21-dabb-4560-a8ee-3558ffa4461b admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:36.043391 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36939c5-8148-4561-a20f-6253068281fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:36.044159 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36939c5-8148-4561-a20f-6253068281fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:36.044309 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36939c5-8148-4561-a20f-6253068281fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:36.044584 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36939c5-8148-4561-a20f-6253068281fd admin admin] PATH_INFO: `/v3/projects/1d66021b7da241fb8ca0a3060f8b7315` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:36.045703 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36939c5-8148-4561-a20f-6253068281fd admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d66021b7da241fb8ca0a3060f8b7315)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:36.047660 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36939c5-8148-4561-a20f-6253068281fd admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:36.092539 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:14:36.093871 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d36939c5-8148-4561-a20f-6253068281fd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d66021b7da241fb8ca0a3060f8b7315', 'request_id': 'req-d36939c5-8148-4561-a20f-6253068281fd'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:36.100889 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d36939c5-8148-4561-a20f-6253068281fd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d66021b7da241fb8ca0a3060f8b7315', 'request_id': 'req-d36939c5-8148-4561-a20f-6253068281fd'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:36.107308 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d36939c5-8148-4561-a20f-6253068281fd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d66021b7da241fb8ca0a3060f8b7315', 'request_id': 'req-d36939c5-8148-4561-a20f-6253068281fd'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:36.120644 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d36939c5-8148-4561-a20f-6253068281fd admin admin] The token cache is being invalidate because project 1d66021b7da241fb8ca0a3060f8b7315 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 04:14:36.120940 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d36939c5-8148-4561-a20f-6253068281fd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:36.121589 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-d36939c5-8148-4561-a20f-6253068281fd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:36.123350 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1319/2630] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 04:14:35 2022] DELETE /identity/v3/projects/1d66021b7da241fb8ca0a3060f8b7315 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:14:36.127984 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32550b1f-418f-4a47-b572-31d93f43ae32 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:36.129387 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-32550b1f-418f-4a47-b572-31d93f43ae32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:14:36.141875 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32550b1f-418f-4a47-b572-31d93f43ae32 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:36.196769 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db11dc3-507a-4dc4-aade-e92a1979a058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:36.197386 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db11dc3-507a-4dc4-aade-e92a1979a058 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:36.197662 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db11dc3-507a-4dc4-aade-e92a1979a058 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:36.197933 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db11dc3-507a-4dc4-aade-e92a1979a058 admin admin] PATH_INFO: `/v3/users/fb321111efc04caba2c08a6bd31c19eb` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:36.199028 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db11dc3-507a-4dc4-aade-e92a1979a058 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb321111efc04caba2c08a6bd31c19eb)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:36.200959 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db11dc3-507a-4dc4-aade-e92a1979a058 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:36.246222 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0db11dc3-507a-4dc4-aade-e92a1979a058 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb321111efc04caba2c08a6bd31c19eb'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:36.254113 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0db11dc3-507a-4dc4-aade-e92a1979a058 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb321111efc04caba2c08a6bd31c19eb'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:36.254769 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0db11dc3-507a-4dc4-aade-e92a1979a058 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb321111efc04caba2c08a6bd31c19eb'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:36.266414 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0db11dc3-507a-4dc4-aade-e92a1979a058 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb321111efc04caba2c08a6bd31c19eb'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:36.273811 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0db11dc3-507a-4dc4-aade-e92a1979a058 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb321111efc04caba2c08a6bd31c19eb'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:36.279730 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1312/2631] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 04:14:36 2022] DELETE /identity/v3/users/fb321111efc04caba2c08a6bd31c19eb => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:14:36.556376 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36939c5-8148-4561-a20f-6253068281fd admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:36.557812 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-d36939c5-8148-4561-a20f-6253068281fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:14:36.574654 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36939c5-8148-4561-a20f-6253068281fd admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:36.630216 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e49a191-5718-441b-b736-21ca87276231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:36.630846 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e49a191-5718-441b-b736-21ca87276231 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:36.631125 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e49a191-5718-441b-b736-21ca87276231 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:36.631398 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e49a191-5718-441b-b736-21ca87276231 admin admin] PATH_INFO: `/v3/projects/c7c957a554d940db8f41d811f5d26b7d` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:36.632530 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e49a191-5718-441b-b736-21ca87276231 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7c957a554d940db8f41d811f5d26b7d)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:36.634423 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e49a191-5718-441b-b736-21ca87276231 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:36.679730 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2e49a191-5718-441b-b736-21ca87276231 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7c957a554d940db8f41d811f5d26b7d', 'request_id': 'req-2e49a191-5718-441b-b736-21ca87276231'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:36.685260 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2e49a191-5718-441b-b736-21ca87276231 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7c957a554d940db8f41d811f5d26b7d', 'request_id': 'req-2e49a191-5718-441b-b736-21ca87276231'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:36.691244 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2e49a191-5718-441b-b736-21ca87276231 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7c957a554d940db8f41d811f5d26b7d', 'request_id': 'req-2e49a191-5718-441b-b736-21ca87276231'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:36.703200 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2e49a191-5718-441b-b736-21ca87276231 admin admin] The token cache is being invalidate because project c7c957a554d940db8f41d811f5d26b7d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 04:14:36.703489 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2e49a191-5718-441b-b736-21ca87276231 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:36.704135 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-2e49a191-5718-441b-b736-21ca87276231 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:36.705800 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1320/2632] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 04:14:36 2022] DELETE /identity/v3/projects/c7c957a554d940db8f41d811f5d26b7d => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:14:36.712396 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7acdeb9-2b09-455f-9843-2479cee6bcdb None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:36.712688 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7acdeb9-2b09-455f-9843-2479cee6bcdb None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:36.712962 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7acdeb9-2b09-455f-9843-2479cee6bcdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:36.732595 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-e7acdeb9-2b09-455f-9843-2479cee6bcdb None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:14:36.733789 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e7acdeb9-2b09-455f-9843-2479cee6bcdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:14:36.773949 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:14:36.794439 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1313/2633] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:14:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:14:36.803454 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc197e-f335-432c-80de-d654f3b6eebf None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:36.803743 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc197e-f335-432c-80de-d654f3b6eebf None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:36.804051 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fc197e-f335-432c-80de-d654f3b6eebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:36.823379 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-b0fc197e-f335-432c-80de-d654f3b6eebf None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:14:36.824602 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-b0fc197e-f335-432c-80de-d654f3b6eebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:14:36.832970 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1321/2634] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:14:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:14:36.837033 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7acdeb9-2b09-455f-9843-2479cee6bcdb None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:36.847528 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7acdeb9-2b09-455f-9843-2479cee6bcdb None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:36.852101 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a76bc36-f6bb-46ac-a394-aad3775a36d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:36.852687 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a76bc36-f6bb-46ac-a394-aad3775a36d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:36.852963 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a76bc36-f6bb-46ac-a394-aad3775a36d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:36.853232 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a76bc36-f6bb-46ac-a394-aad3775a36d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:36.853869 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a76bc36-f6bb-46ac-a394-aad3775a36d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Nov 02 04:14:36.854168 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a76bc36-f6bb-46ac-a394-aad3775a36d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:36.855730 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a76bc36-f6bb-46ac-a394-aad3775a36d5 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:36.867039 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1314/2635] 10.222.0.26 () {58 vars in 1153 bytes} [Wed Nov 2 04:14:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Nov 02 04:14:36.871058 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fc197e-f335-432c-80de-d654f3b6eebf None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:36.880539 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fc197e-f335-432c-80de-d654f3b6eebf None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:36.885101 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9140a960-dede-43e1-9a05-dcc0d4e9c0d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:36.885690 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140a960-dede-43e1-9a05-dcc0d4e9c0d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:36.885966 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140a960-dede-43e1-9a05-dcc0d4e9c0d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:36.886236 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140a960-dede-43e1-9a05-dcc0d4e9c0d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:36.886855 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9140a960-dede-43e1-9a05-dcc0d4e9c0d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:36.888761 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9140a960-dede-43e1-9a05-dcc0d4e9c0d8 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:36.916458 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1322/2636] 10.222.0.26 () {60 vars in 1153 bytes} [Wed Nov 2 04:14:36 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:14:36.920457 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a76bc36-f6bb-46ac-a394-aad3775a36d5 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:36.930178 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a76bc36-f6bb-46ac-a394-aad3775a36d5 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:36.985951 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab4499c-60a8-4534-a5c3-cd42d0c41184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:36.986556 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab4499c-60a8-4534-a5c3-cd42d0c41184 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:36.986839 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab4499c-60a8-4534-a5c3-cd42d0c41184 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:36.987114 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab4499c-60a8-4534-a5c3-cd42d0c41184 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:36.987749 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab4499c-60a8-4534-a5c3-cd42d0c41184 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:36.989734 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab4499c-60a8-4534-a5c3-cd42d0c41184 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:37.010497 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1315/2637] 10.222.0.26 () {60 vars in 1144 bytes} [Wed Nov 2 04:14:36 2022] POST /identity/v3/users => generated 293 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Nov 02 04:14:37.014347 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9140a960-dede-43e1-9a05-dcc0d4e9c0d8 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:37.025246 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9140a960-dede-43e1-9a05-dcc0d4e9c0d8 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:37.029875 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6562d28d-f9e7-4147-8aef-7e0287a16eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:37.030471 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562d28d-f9e7-4147-8aef-7e0287a16eb2 admin admin] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:37.030749 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562d28d-f9e7-4147-8aef-7e0287a16eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:37.031021 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6562d28d-f9e7-4147-8aef-7e0287a16eb2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:37.031657 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6562d28d-f9e7-4147-8aef-7e0287a16eb2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:37.033905 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6562d28d-f9e7-4147-8aef-7e0287a16eb2 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:37.043626 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1323/2638] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:14:37.047578 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab4499c-60a8-4534-a5c3-cd42d0c41184 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:37.057650 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab4499c-60a8-4534-a5c3-cd42d0c41184 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:37.062285 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe428bfd-5007-4760-b7f2-2492598ffe60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:37.062879 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe428bfd-5007-4760-b7f2-2492598ffe60 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:37.063159 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe428bfd-5007-4760-b7f2-2492598ffe60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:37.063433 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe428bfd-5007-4760-b7f2-2492598ffe60 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:37.064120 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe428bfd-5007-4760-b7f2-2492598ffe60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:37.065735 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe428bfd-5007-4760-b7f2-2492598ffe60 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:37.076141 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1316/2639] 10.222.0.26 () {58 vars in 1122 bytes} [Wed Nov 2 04:14:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Nov 02 04:14:37.080194 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6562d28d-f9e7-4147-8aef-7e0287a16eb2 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:37.090637 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6562d28d-f9e7-4147-8aef-7e0287a16eb2 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:37.095368 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47400685-f0e3-4563-9d3d-7d15278fe35f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:37.096039 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47400685-f0e3-4563-9d3d-7d15278fe35f admin admin] REQUEST_METHOD: `PUT` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:37.096324 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47400685-f0e3-4563-9d3d-7d15278fe35f admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:37.096603 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47400685-f0e3-4563-9d3d-7d15278fe35f admin admin] PATH_INFO: `/v3/projects/d026c2a3c1c34e99a7b00e7e3123c30d/users/33dd0f289b784c8d805f857b5d22b4fe/roles/7a6df2ce66694822bce7983f7d988da5` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:37.111611 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47400685-f0e3-4563-9d3d-7d15278fe35f admin admin] RBAC: Authorizing `identity:create_grant(project_id=d026c2a3c1c34e99a7b00e7e3123c30d, user_id=33dd0f289b784c8d805f857b5d22b4fe, role_id=7a6df2ce66694822bce7983f7d988da5)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:37.114354 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47400685-f0e3-4563-9d3d-7d15278fe35f admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:37.126998 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1324/2640] 10.222.0.26 () {60 vars in 1483 bytes} [Wed Nov 2 04:14:37 2022] PUT /identity/v3/projects/d026c2a3c1c34e99a7b00e7e3123c30d/users/33dd0f289b784c8d805f857b5d22b4fe/roles/7a6df2ce66694822bce7983f7d988da5 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:14:37.139323 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f2015-4a92-4b0e-bd5c-fce9baf46c8a None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:37.139861 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f2015-4a92-4b0e-bd5c-fce9baf46c8a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:37.139861 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471f2015-4a92-4b0e-bd5c-fce9baf46c8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:37.170397 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-471f2015-4a92-4b0e-bd5c-fce9baf46c8a None None] MFA Rules not processed for user `33dd0f289b784c8d805f857b5d22b4fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:14:37.173201 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-471f2015-4a92-4b0e-bd5c-fce9baf46c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:14:37.238673 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1317/2641] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:14:37 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:14:37.250492 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47400685-f0e3-4563-9d3d-7d15278fe35f admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:37.252194 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-47400685-f0e3-4563-9d3d-7d15278fe35f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:14:37.257971 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:14:37.311821 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47400685-f0e3-4563-9d3d-7d15278fe35f admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:37.315416 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71994b37-61e3-4d6e-8bf9-5f1f1d712b58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'df541db335eb42dc8b36cc858fa7c22f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:37.316562 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71994b37-61e3-4d6e-8bf9-5f1f1d712b58 service cinder] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:37.316836 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71994b37-61e3-4d6e-8bf9-5f1f1d712b58 service cinder] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:37.317109 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71994b37-61e3-4d6e-8bf9-5f1f1d712b58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:37.325206 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71994b37-61e3-4d6e-8bf9-5f1f1d712b58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:37.326611 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71994b37-61e3-4d6e-8bf9-5f1f1d712b58 service cinder] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:37.335915 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1325/2642] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:14:37 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:14:39.880811 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471f2015-4a92-4b0e-bd5c-fce9baf46c8a None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:39.922135 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:14:39.942343 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471f2015-4a92-4b0e-bd5c-fce9baf46c8a None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:39.946762 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb4474a-f83f-4d76-8fc7-53fd723e2cde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:39.947383 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb4474a-f83f-4d76-8fc7-53fd723e2cde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:39.947662 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb4474a-f83f-4d76-8fc7-53fd723e2cde admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:39.947930 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb4474a-f83f-4d76-8fc7-53fd723e2cde admin admin] PATH_INFO: `/v3/users/33dd0f289b784c8d805f857b5d22b4fe` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:39.949044 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb4474a-f83f-4d76-8fc7-53fd723e2cde admin admin] RBAC: Authorizing `identity:delete_user(user_id=33dd0f289b784c8d805f857b5d22b4fe)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:39.950931 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb4474a-f83f-4d76-8fc7-53fd723e2cde admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:39.996196 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2cb4474a-f83f-4d76-8fc7-53fd723e2cde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33dd0f289b784c8d805f857b5d22b4fe'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:40.003381 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2cb4474a-f83f-4d76-8fc7-53fd723e2cde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33dd0f289b784c8d805f857b5d22b4fe'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:40.004093 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2cb4474a-f83f-4d76-8fc7-53fd723e2cde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33dd0f289b784c8d805f857b5d22b4fe'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:40.015824 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2cb4474a-f83f-4d76-8fc7-53fd723e2cde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33dd0f289b784c8d805f857b5d22b4fe'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:40.023486 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2cb4474a-f83f-4d76-8fc7-53fd723e2cde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33dd0f289b784c8d805f857b5d22b4fe'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:40.029363 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1318/2643] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 04:14:39 2022] DELETE /identity/v3/users/33dd0f289b784c8d805f857b5d22b4fe => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:14:40.033931 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc66fe-2b6f-4fa5-b846-d02c7c833363 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:40.034425 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc66fe-2b6f-4fa5-b846-d02c7c833363 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:40.034530 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cc66fe-2b6f-4fa5-b846-d02c7c833363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:40.054320 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-61cc66fe-2b6f-4fa5-b846-d02c7c833363 None None] MFA Rules not processed for user `cc903821fbc442d9b4a30f92f10f1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:14:40.055518 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-61cc66fe-2b6f-4fa5-b846-d02c7c833363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:14:40.112965 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1326/2644] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:14:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:14:40.122685 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb4474a-f83f-4d76-8fc7-53fd723e2cde admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:40.124104 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2cb4474a-f83f-4d76-8fc7-53fd723e2cde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:14:40.179625 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb4474a-f83f-4d76-8fc7-53fd723e2cde admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:40.183243 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9ff664-1dc6-427d-bdbd-cee607abd3a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:40.183835 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ff664-1dc6-427d-bdbd-cee607abd3a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:40.184137 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ff664-1dc6-427d-bdbd-cee607abd3a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:40.184406 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ff664-1dc6-427d-bdbd-cee607abd3a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:40.193687 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9ff664-1dc6-427d-bdbd-cee607abd3a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:40.195091 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9ff664-1dc6-427d-bdbd-cee607abd3a8 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:40.204238 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1319/2645] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:14:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:14:40.471738 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cc66fe-2b6f-4fa5-b846-d02c7c833363 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:14:40.473259 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-61cc66fe-2b6f-4fa5-b846-d02c7c833363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:14:40.485760 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cc66fe-2b6f-4fa5-b846-d02c7c833363 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:14:40.490920 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e949e-a949-4685-91ba-5fedb9df4544 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:14:40.491591 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e949e-a949-4685-91ba-5fedb9df4544 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:14:40.491894 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e949e-a949-4685-91ba-5fedb9df4544 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:14:40.492197 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e949e-a949-4685-91ba-5fedb9df4544 admin admin] PATH_INFO: `/v3/projects/d026c2a3c1c34e99a7b00e7e3123c30d` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:14:40.494406 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4e949e-a949-4685-91ba-5fedb9df4544 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d026c2a3c1c34e99a7b00e7e3123c30d)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:14:40.496459 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4e949e-a949-4685-91ba-5fedb9df4544 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:14:40.544972 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ff4e949e-a949-4685-91ba-5fedb9df4544 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd026c2a3c1c34e99a7b00e7e3123c30d', 'request_id': 'req-ff4e949e-a949-4685-91ba-5fedb9df4544'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:40.551989 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ff4e949e-a949-4685-91ba-5fedb9df4544 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd026c2a3c1c34e99a7b00e7e3123c30d', 'request_id': 'req-ff4e949e-a949-4685-91ba-5fedb9df4544'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:40.558307 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ff4e949e-a949-4685-91ba-5fedb9df4544 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd026c2a3c1c34e99a7b00e7e3123c30d', 'request_id': 'req-ff4e949e-a949-4685-91ba-5fedb9df4544'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:40.571197 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ff4e949e-a949-4685-91ba-5fedb9df4544 admin admin] The token cache is being invalidate because project d026c2a3c1c34e99a7b00e7e3123c30d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 04:14:40.571485 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ff4e949e-a949-4685-91ba-5fedb9df4544 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:40.572151 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-ff4e949e-a949-4685-91ba-5fedb9df4544 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:14:40.574055 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1327/2646] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 04:14:40 2022] DELETE /identity/v3/projects/d026c2a3c1c34e99a7b00e7e3123c30d => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:14:50.200229 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=61,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:14:50.568111 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=79,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:15:10.354336 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9ff664-1dc6-427d-bdbd-cee607abd3a8 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:15:10.355837 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-cb9ff664-1dc6-427d-bdbd-cee607abd3a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:15:10.376382 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9ff664-1dc6-427d-bdbd-cee607abd3a8 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:15:10.424464 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e15b08-2bb8-404e-b065-0978493e2dd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:15:10.425097 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e15b08-2bb8-404e-b065-0978493e2dd4 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:15:10.425382 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e15b08-2bb8-404e-b065-0978493e2dd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:15:10.425657 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e15b08-2bb8-404e-b065-0978493e2dd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:15:10.427376 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-41e15b08-2bb8-404e-b065-0978493e2dd4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:15:10.436572 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e15b08-2bb8-404e-b065-0978493e2dd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:15:10.437979 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e15b08-2bb8-404e-b065-0978493e2dd4 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:15:10.446416 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1320/2647] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:15:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:15:12.525252 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e949e-a949-4685-91ba-5fedb9df4544 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:15:12.526779 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-ff4e949e-a949-4685-91ba-5fedb9df4544 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:15:12.539697 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e949e-a949-4685-91ba-5fedb9df4544 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:15:12.543277 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f7803f-46e0-41f9-a50c-ee83a175ffdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:15:12.543878 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f7803f-46e0-41f9-a50c-ee83a175ffdc service neutron] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:15:12.544260 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f7803f-46e0-41f9-a50c-ee83a175ffdc service neutron] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:15:12.544533 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f7803f-46e0-41f9-a50c-ee83a175ffdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:15:12.546291 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-42f7803f-46e0-41f9-a50c-ee83a175ffdc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:15:12.637810 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f7803f-46e0-41f9-a50c-ee83a175ffdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:15:12.639135 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f7803f-46e0-41f9-a50c-ee83a175ffdc service neutron] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:15:12.658881 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1328/2648] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:15:12 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:15:15.423426 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e15b08-2bb8-404e-b065-0978493e2dd4 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:15:15.425646 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-41e15b08-2bb8-404e-b065-0978493e2dd4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:15:15.442238 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e15b08-2bb8-404e-b065-0978493e2dd4 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:15:15.450292 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5833ced1-2501-4635-9141-062f6a3e7ac0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:15:15.451007 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5833ced1-2501-4635-9141-062f6a3e7ac0 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:15:15.451362 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5833ced1-2501-4635-9141-062f6a3e7ac0 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:15:15.451726 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5833ced1-2501-4635-9141-062f6a3e7ac0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:15:15.453609 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5833ced1-2501-4635-9141-062f6a3e7ac0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:15:15.467503 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5833ced1-2501-4635-9141-062f6a3e7ac0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:15:15.469726 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5833ced1-2501-4635-9141-062f6a3e7ac0 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:15:15.485028 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1321/2649] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:15:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:15:17.396170 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f65360-7935-4d7b-ad45-21bfb1ea3420 None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:15:17.396665 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f65360-7935-4d7b-ad45-21bfb1ea3420 None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:15:17.396769 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f65360-7935-4d7b-ad45-21bfb1ea3420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:15:17.415250 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-11f65360-7935-4d7b-ad45-21bfb1ea3420 None None] MFA Rules not processed for user `390f0d31f2ee42d9aba3922cce6e78ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:15:17.416485 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-11f65360-7935-4d7b-ad45-21bfb1ea3420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:15:17.470042 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1329/2650] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:15:17 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:15:17.775356 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5833ced1-2501-4635-9141-062f6a3e7ac0 service nova] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:15:17.785428 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5833ced1-2501-4635-9141-062f6a3e7ac0 service nova] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:15:17.789030 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7378a31b-7890-4787-a359-c631a327204a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:15:17.789624 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7378a31b-7890-4787-a359-c631a327204a service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:15:17.789902 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7378a31b-7890-4787-a359-c631a327204a service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:15:17.790174 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7378a31b-7890-4787-a359-c631a327204a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:15:17.791909 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7378a31b-7890-4787-a359-c631a327204a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:15:17.802016 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7378a31b-7890-4787-a359-c631a327204a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:15:17.803415 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7378a31b-7890-4787-a359-c631a327204a service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:15:17.811364 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1322/2651] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:15:17 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:15:19.266201 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f65360-7935-4d7b-ad45-21bfb1ea3420 None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:15:19.267595 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-11f65360-7935-4d7b-ad45-21bfb1ea3420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:15:19.336601 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f65360-7935-4d7b-ad45-21bfb1ea3420 None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:15:19.341597 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0d0825-6a75-401f-99c4-287fe569b27d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4b9bd741574ddabadd4b7651ebc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:15:19.342213 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0d0825-6a75-401f-99c4-287fe569b27d service placement] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:15:19.342609 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0d0825-6a75-401f-99c4-287fe569b27d service placement] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:15:19.342890 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0d0825-6a75-401f-99c4-287fe569b27d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:15:19.344629 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-7b0d0825-6a75-401f-99c4-287fe569b27d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:15:19.353507 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=100 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:15:19.358026 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0d0825-6a75-401f-99c4-287fe569b27d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:15:19.359626 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0d0825-6a75-401f-99c4-287fe569b27d service placement] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:15:19.370785 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1330/2652] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:15:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:15:27.807712 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=42 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:15:29.366323 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:15:36.801489 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7378a31b-7890-4787-a359-c631a327204a service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:15:36.817636 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7378a31b-7890-4787-a359-c631a327204a service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:15:36.823240 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f894e4-493b-4cd6-9746-33d21e0e34cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:15:36.823847 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f894e4-493b-4cd6-9746-33d21e0e34cb service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:15:36.824162 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f894e4-493b-4cd6-9746-33d21e0e34cb service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:15:36.824430 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f894e4-493b-4cd6-9746-33d21e0e34cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:15:36.826300 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-09f894e4-493b-4cd6-9746-33d21e0e34cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:15:36.840152 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f894e4-493b-4cd6-9746-33d21e0e34cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:15:36.842682 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f894e4-493b-4cd6-9746-33d21e0e34cb service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:15:36.858678 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1323/2653] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:15:36 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:15:46.853590 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=6 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:17:26.578154 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb779a60-cf6f-4e7f-89ed-895e0681d3cb None None] REQUEST_METHOD: `POST` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:17:26.578662 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb779a60-cf6f-4e7f-89ed-895e0681d3cb None None] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:17:26.578662 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb779a60-cf6f-4e7f-89ed-895e0681d3cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:17:26.597669 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.auth.core [None req-bb779a60-cf6f-4e7f-89ed-895e0681d3cb None None] MFA Rules not processed for user `390f0d31f2ee42d9aba3922cce6e78ee`. Rule list: `[]` (Enabled: `True`). {{(pid=81504) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Nov 02 04:17:26.598847 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-bb779a60-cf6f-4e7f-89ed-895e0681d3cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:17:26.605507 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1331/2654] 10.222.0.26 () {58 vars in 958 bytes} [Wed Nov 2 04:17:26 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Nov 02 04:17:34.419154 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f894e4-493b-4cd6-9746-33d21e0e34cb service nova] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:17:34.430507 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f894e4-493b-4cd6-9746-33d21e0e34cb service nova] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:17:34.436045 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c61d5-75df-443d-997d-c084be8093c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:17:34.436638 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c61d5-75df-443d-997d-c084be8093c9 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:17:34.436917 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c61d5-75df-443d-997d-c084be8093c9 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:17:34.437194 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c61d5-75df-443d-997d-c084be8093c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:17:34.438882 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ab7c61d5-75df-443d-997d-c084be8093c9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:17:34.449238 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7c61d5-75df-443d-997d-c084be8093c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:17:34.450809 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7c61d5-75df-443d-997d-c084be8093c9 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:17:34.459942 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1324/2655] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:17:34 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:17:36.592372 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=3 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:17:44.455501 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=6 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:18:03.847134 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb779a60-cf6f-4e7f-89ed-895e0681d3cb None None] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:18:03.848551 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-bb779a60-cf6f-4e7f-89ed-895e0681d3cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:18:03.862982 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb779a60-cf6f-4e7f-89ed-895e0681d3cb None None] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:18:03.867824 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aca068-c2e6-4b70-87ae-2b6ae6c340f9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4b9bd741574ddabadd4b7651ebc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:18:03.868449 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aca068-c2e6-4b70-87ae-2b6ae6c340f9 service placement] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:18:03.868729 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aca068-c2e6-4b70-87ae-2b6ae6c340f9 service placement] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:18:03.868996 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aca068-c2e6-4b70-87ae-2b6ae6c340f9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:18:03.870634 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-83aca068-c2e6-4b70-87ae-2b6ae6c340f9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:18:03.881999 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aca068-c2e6-4b70-87ae-2b6ae6c340f9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:18:03.883731 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aca068-c2e6-4b70-87ae-2b6ae6c340f9 service placement] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:18:03.893993 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1332/2656] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:18:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:18:13.889894 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=6 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:19:16.629373 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c61d5-75df-443d-997d-c084be8093c9 service nova] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:19:16.639271 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c61d5-75df-443d-997d-c084be8093c9 service nova] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:19:16.642877 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3261b68-c476-4947-828a-10ebda334b4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:19:16.643469 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3261b68-c476-4947-828a-10ebda334b4e service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:19:16.643746 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3261b68-c476-4947-828a-10ebda334b4e service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:19:16.644053 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3261b68-c476-4947-828a-10ebda334b4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:19:16.645749 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c3261b68-c476-4947-828a-10ebda334b4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:19:16.660160 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3261b68-c476-4947-828a-10ebda334b4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:19:16.661560 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3261b68-c476-4947-828a-10ebda334b4e service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:19:16.672551 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1325/2657] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:19:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:19:17.876782 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aca068-c2e6-4b70-87ae-2b6ae6c340f9 service placement] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:19:17.893265 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aca068-c2e6-4b70-87ae-2b6ae6c340f9 service placement] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:19:17.899158 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4256ade-963d-4086-b7ca-77394473d891 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '44961b06151c4da382fcd54dd1f1e146', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'ResellerAdmin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:19:17.899768 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4256ade-963d-4086-b7ca-77394473d891 service nova] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:19:17.900307 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4256ade-963d-4086-b7ca-77394473d891 service nova] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:19:17.900565 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4256ade-963d-4086-b7ca-77394473d891 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:19:17.903034 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-c4256ade-963d-4086-b7ca-77394473d891 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:19:17.916768 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4256ade-963d-4086-b7ca-77394473d891 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:19:17.918357 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4256ade-963d-4086-b7ca-77394473d891 service nova] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:19:17.930153 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1333/2658] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:19:17 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:19:26.667539 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=6 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:19:27.522985 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3261b68-c476-4947-828a-10ebda334b4e service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:19:27.532004 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3261b68-c476-4947-828a-10ebda334b4e service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:19:27.535601 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6d8a39-334b-4167-8f37-0e4b9e2df14b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f07e1b51b00f42b7b74c3689268635cb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:19:27.536227 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d8a39-334b-4167-8f37-0e4b9e2df14b service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:19:27.536511 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d8a39-334b-4167-8f37-0e4b9e2df14b service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:19:27.536784 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d8a39-334b-4167-8f37-0e4b9e2df14b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:19:27.538505 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5b6d8a39-334b-4167-8f37-0e4b9e2df14b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:19:27.622244 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6d8a39-334b-4167-8f37-0e4b9e2df14b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:19:27.623937 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6d8a39-334b-4167-8f37-0e4b9e2df14b service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:19:27.643060 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1326/2659] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:19:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:19:27.925241 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=6 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:19:30.656213 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4256ade-963d-4086-b7ca-77394473d891 service nova] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:19:30.672166 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4256ade-963d-4086-b7ca-77394473d891 service nova] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:19:30.677590 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ac2af3-6132-4d9b-8ce0-8981718815bd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4b9bd741574ddabadd4b7651ebc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:19:30.678212 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac2af3-6132-4d9b-8ce0-8981718815bd service placement] REQUEST_METHOD: `GET` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:19:30.678488 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac2af3-6132-4d9b-8ce0-8981718815bd service placement] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:19:30.678758 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ac2af3-6132-4d9b-8ce0-8981718815bd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:19:30.680623 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-45ac2af3-6132-4d9b-8ce0-8981718815bd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:19:30.695649 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ac2af3-6132-4d9b-8ce0-8981718815bd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:19:30.702818 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ac2af3-6132-4d9b-8ce0-8981718815bd service placement] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:19:30.723186 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1334/2660] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:19:30 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:19:33.874827 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6d8a39-334b-4167-8f37-0e4b9e2df14b service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:19:33.876439 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5b6d8a39-334b-4167-8f37-0e4b9e2df14b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:19:33.888767 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6d8a39-334b-4167-8f37-0e4b9e2df14b service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:19:33.893557 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84934e2-7dc7-43fd-a127-276ff9f39667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:19:33.894189 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84934e2-7dc7-43fd-a127-276ff9f39667 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:19:33.894470 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84934e2-7dc7-43fd-a127-276ff9f39667 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:19:33.894743 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84934e2-7dc7-43fd-a127-276ff9f39667 admin admin] PATH_INFO: `/v3/users/390f0d31f2ee42d9aba3922cce6e78ee` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:19:33.895856 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84934e2-7dc7-43fd-a127-276ff9f39667 admin admin] RBAC: Authorizing `identity:delete_user(user_id=390f0d31f2ee42d9aba3922cce6e78ee)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:19:33.897839 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84934e2-7dc7-43fd-a127-276ff9f39667 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:19:33.944279 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-c84934e2-7dc7-43fd-a127-276ff9f39667 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '390f0d31f2ee42d9aba3922cce6e78ee'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:19:33.951658 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-c84934e2-7dc7-43fd-a127-276ff9f39667 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '390f0d31f2ee42d9aba3922cce6e78ee'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:19:33.952337 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-c84934e2-7dc7-43fd-a127-276ff9f39667 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '390f0d31f2ee42d9aba3922cce6e78ee'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:19:33.964243 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-c84934e2-7dc7-43fd-a127-276ff9f39667 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '390f0d31f2ee42d9aba3922cce6e78ee'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:19:33.972020 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-c84934e2-7dc7-43fd-a127-276ff9f39667 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '390f0d31f2ee42d9aba3922cce6e78ee'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:19:33.977907 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1327/2661] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 04:19:33 2022] DELETE /identity/v3/users/390f0d31f2ee42d9aba3922cce6e78ee => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:19:34.154152 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ac2af3-6132-4d9b-8ce0-8981718815bd service placement] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:19:34.165527 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ac2af3-6132-4d9b-8ce0-8981718815bd service placement] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:19:34.227442 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4de78e3-2840-4f83-87e8-a6e474150e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:19:34.228094 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de78e3-2840-4f83-87e8-a6e474150e56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:19:34.228374 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de78e3-2840-4f83-87e8-a6e474150e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:19:34.228644 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4de78e3-2840-4f83-87e8-a6e474150e56 admin admin] PATH_INFO: `/v3/projects/143ef6c3d11b42c595e2fda02cc5a1b8` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:19:34.229743 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4de78e3-2840-4f83-87e8-a6e474150e56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=143ef6c3d11b42c595e2fda02cc5a1b8)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:19:34.231671 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4de78e3-2840-4f83-87e8-a6e474150e56 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:19:34.276538 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b4de78e3-2840-4f83-87e8-a6e474150e56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '143ef6c3d11b42c595e2fda02cc5a1b8', 'request_id': 'req-b4de78e3-2840-4f83-87e8-a6e474150e56'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:19:34.282694 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b4de78e3-2840-4f83-87e8-a6e474150e56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '143ef6c3d11b42c595e2fda02cc5a1b8', 'request_id': 'req-b4de78e3-2840-4f83-87e8-a6e474150e56'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:19:34.288585 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b4de78e3-2840-4f83-87e8-a6e474150e56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '143ef6c3d11b42c595e2fda02cc5a1b8', 'request_id': 'req-b4de78e3-2840-4f83-87e8-a6e474150e56'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:19:34.301581 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b4de78e3-2840-4f83-87e8-a6e474150e56 admin admin] The token cache is being invalidate because project 143ef6c3d11b42c595e2fda02cc5a1b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 04:19:34.301873 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b4de78e3-2840-4f83-87e8-a6e474150e56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:19:34.302493 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-b4de78e3-2840-4f83-87e8-a6e474150e56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:19:34.303989 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1335/2662] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 04:19:34 2022] DELETE /identity/v3/projects/143ef6c3d11b42c595e2fda02cc5a1b8 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:19:34.308175 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84934e2-7dc7-43fd-a127-276ff9f39667 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:19:34.309788 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c84934e2-7dc7-43fd-a127-276ff9f39667 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:19:34.322595 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84934e2-7dc7-43fd-a127-276ff9f39667 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:19:34.356913 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:19:34.381252 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ffbb12-2f19-40e5-8681-e8883f79bc73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:19:34.381887 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ffbb12-2f19-40e5-8681-e8883f79bc73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:19:34.382167 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ffbb12-2f19-40e5-8681-e8883f79bc73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:19:34.382442 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ffbb12-2f19-40e5-8681-e8883f79bc73 admin admin] PATH_INFO: `/v3/users/1c9e1f35c6a948c593ceb292f7b4e3c4` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:19:34.383575 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ffbb12-2f19-40e5-8681-e8883f79bc73 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c9e1f35c6a948c593ceb292f7b4e3c4)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:19:34.385519 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ffbb12-2f19-40e5-8681-e8883f79bc73 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:19:34.430697 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-74ffbb12-2f19-40e5-8681-e8883f79bc73 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c9e1f35c6a948c593ceb292f7b4e3c4'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:19:34.437612 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-74ffbb12-2f19-40e5-8681-e8883f79bc73 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c9e1f35c6a948c593ceb292f7b4e3c4'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:19:34.438277 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-74ffbb12-2f19-40e5-8681-e8883f79bc73 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c9e1f35c6a948c593ceb292f7b4e3c4'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:19:34.449851 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-74ffbb12-2f19-40e5-8681-e8883f79bc73 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c9e1f35c6a948c593ceb292f7b4e3c4'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:19:34.456829 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-74ffbb12-2f19-40e5-8681-e8883f79bc73 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c9e1f35c6a948c593ceb292f7b4e3c4'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:19:34.462571 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1328/2663] 10.222.0.26 () {58 vars in 1224 bytes} [Wed Nov 2 04:19:34 2022] DELETE /identity/v3/users/1c9e1f35c6a948c593ceb292f7b4e3c4 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Nov 02 04:19:34.637572 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4de78e3-2840-4f83-87e8-a6e474150e56 admin admin] Authenticating user token {{(pid=81504) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:19:34.638996 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.fernet_utils [None req-b4de78e3-2840-4f83-87e8-a6e474150e56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81504) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:19:34.651875 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4de78e3-2840-4f83-87e8-a6e474150e56 admin admin] Validating token access rules against request {{(pid=81504) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:19:34.709527 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e164a48f-1b3e-4ca9-a2a7-4f89b43b2e41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc903821fbc442d9b4a30f92f10f1b17', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '90ca3cc9c54d44feb93e6ed209b99483', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81504) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:19:34.710162 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e164a48f-1b3e-4ca9-a2a7-4f89b43b2e41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:19:34.710439 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e164a48f-1b3e-4ca9-a2a7-4f89b43b2e41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:19:34.710711 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e164a48f-1b3e-4ca9-a2a7-4f89b43b2e41 admin admin] PATH_INFO: `/v3/projects/f48a1a56ed8e43f79571a3bbc74045ac` {{(pid=81504) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:19:34.711852 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e164a48f-1b3e-4ca9-a2a7-4f89b43b2e41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f48a1a56ed8e43f79571a3bbc74045ac)` {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:19:34.714021 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e164a48f-1b3e-4ca9-a2a7-4f89b43b2e41 admin admin] RBAC: Authorization granted {{(pid=81504) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:19:34.734995 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:19:34.761835 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e164a48f-1b3e-4ca9-a2a7-4f89b43b2e41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f48a1a56ed8e43f79571a3bbc74045ac', 'request_id': 'req-e164a48f-1b3e-4ca9-a2a7-4f89b43b2e41'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:19:34.767290 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e164a48f-1b3e-4ca9-a2a7-4f89b43b2e41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f48a1a56ed8e43f79571a3bbc74045ac', 'request_id': 'req-e164a48f-1b3e-4ca9-a2a7-4f89b43b2e41'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:19:34.773296 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e164a48f-1b3e-4ca9-a2a7-4f89b43b2e41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f48a1a56ed8e43f79571a3bbc74045ac', 'request_id': 'req-e164a48f-1b3e-4ca9-a2a7-4f89b43b2e41'} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:19:34.785032 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e164a48f-1b3e-4ca9-a2a7-4f89b43b2e41 admin admin] The token cache is being invalidate because project f48a1a56ed8e43f79571a3bbc74045ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81504) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Nov 02 04:19:34.785323 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e164a48f-1b3e-4ca9-a2a7-4f89b43b2e41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:19:34.785958 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG keystone.notifications [None req-e164a48f-1b3e-4ca9-a2a7-4f89b43b2e41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81504) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Nov 02 04:19:34.787445 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: [pid: 81504|app: 0|req: 1336/2664] 10.222.0.26 () {58 vars in 1233 bytes} [Wed Nov 2 04:19:34 2022] DELETE /identity/v3/projects/f48a1a56ed8e43f79571a3bbc74045ac => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Nov 02 04:19:44.459222 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=36,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:19:44.782029 dv-cinder-860209-6-iscsi devstack@keystone.service[81504]: DEBUG dbcounter [-] [81504] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81504) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Nov 02 04:23:10.882418 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ffbb12-2f19-40e5-8681-e8883f79bc73 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Nov 02 04:23:10.884569 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-74ffbb12-2f19-40e5-8681-e8883f79bc73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:23:11.013983 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ffbb12-2f19-40e5-8681-e8883f79bc73 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Nov 02 04:23:11.069856 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f2df77-e5ca-4a59-9f5c-f44d6ef50d39 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9b4b9bd741574ddabadd4b7651ebc885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0b5e7017ac64ae6ab62c1a05aa54b72', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Nov 02 04:23:11.070455 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f2df77-e5ca-4a59-9f5c-f44d6ef50d39 service placement] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Nov 02 04:23:11.070730 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f2df77-e5ca-4a59-9f5c-f44d6ef50d39 service placement] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Nov 02 04:23:11.070998 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f2df77-e5ca-4a59-9f5c-f44d6ef50d39 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Nov 02 04:23:11.073279 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d8f2df77-e5ca-4a59-9f5c-f44d6ef50d39 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Nov 02 04:23:11.084094 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f2df77-e5ca-4a59-9f5c-f44d6ef50d39 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Nov 02 04:23:11.085653 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f2df77-e5ca-4a59-9f5c-f44d6ef50d39 service placement] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Nov 02 04:23:11.096317 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1329/2665] 10.222.0.26 () {60 vars in 1365 bytes} [Wed Nov 2 04:23:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Nov 02 04:23:21.091887 dv-cinder-860209-6-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=84 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}